Documentation / WirisQuizzes

  • Demos
  • Visit our website
  • Contact us
  • MathType

    • Wiris Quizzes

      • Learning Lemur

        • CalcMe

          • MathPlayer

            • Store FAQ

              • VPAT for the electronic documentation

                • MathFlow

                  • BF FAQ

                    • Miscellaneous

                      • Wiris Integrations

                        • Home
                        • Wiris Quizzes
                        • Validation Options
                        • Numbers representation, tolerance and precision
                        • Numbers representation, tolerance and precision

                        Precision

                        Reading time: 1min


                        Precision is the number of significant figures or decimal places of variables shown after calculations. Precision affects the decimal numbers shown in wordings, feedback, and so. When rounding, the rule used for tie-breaking is half-up.

                        Consider the number 12.345

                        Precision

                        Rounded to significant figures

                        Rounded to decimal places

                        6 12.345     12.345000
                        5 12.345     12.34500
                        4 12.35     12.3450
                        3 12.3     12.345
                        2 12     12.35
                        1 12     12.3

                        Precision is not the allowed deviation between students and correct answers; that is Tolerance. But precision can harm the comparison if set improperly. Precision affects the correct answer, how it's shown to the student, and also the actual value. The student answer will be compared with the correct answer that has been rounded using precision. If the precision number is smaller than the tolerance number, the correct answer will be rounded too much, probably outside the tolerance range. So remember to set the precision number always more significant than the tolerance number.

                        Was this article helpful?

                        Yes
                        No
                        Give feedback about this article

                        Related Articles

                        • Numbers representation, tolerance and precision
                        • Integers and decimal numbers
                        • Tolerance
                        • Notation
                        • Concrete number of decimals as assertion

                        Making people’s STEM work more meaningful

                        MathType

                        • MathType for Office Tools
                        • MathType for Mac
                        • MathType for Microsoft 365
                        • MathType for Google Workspace
                        • MathType for LMS
                        • MathType for XML Editors
                        • Arabic notation
                        • Our products accessibility
                        • MathType is online

                        WirisQuizzes

                        Learning Lemur

                        Solutions for Education

                        • Blackboard Learn
                        • Brightspace by D2L
                        • Canvas
                        • Google Classroom
                        • Moodle
                        • Schoology

                        Solutions for Publishing Houses

                        Solutions for Technical Writers

                        Solutions for Chemistry

                        Integrations

                        • HTML Editors
                        • MathType in WordPress

                        Pricing

                        Company

                        Careers

                        Blog

                        Contact Us

                        Buy Now

                        Plugin Downloads

                        © Wiris 2025

                        • Cookie Settings
                        • Cookie Policy
                        • Terms of Use
                        • Privacy Policy / GDPR
                        • Student Data Privacy
                        • Compliance
                        • Powered by Helpjuice
                        Expand