Documentation / WirisQuizzes

  • Demos
  • Visit our website
  • Contact us
  • MathType

    • Wiris Quizzes

      • Learning Lemur

        • CalcMe

          • MathPlayer

            • Store FAQ

              • VPAT for the electronic documentation

                • MathFlow

                  • BF FAQ

                    • Home
                    • Wiris Quizzes
                    • Validation Options
                    • Numbers representation, tolerance and precision
                    • Numbers representation, tolerance and precision

                    Numbers representation, tolerance and precision

                    Reading time: 5min

                    On this page, you will find out how integers and decimal numbers are treated in CalcMe. You can also find a detailed explanation of Precision and Tolerance. You can choose between different options for the Notation used when displaying numbers. Finally, you can specify answers with repeated decimals or a concrete number of decimal places or significant figures.

                    Integers and decimal numbers

                    CalcMe does calculations in exact mode and only falls to the approximate mode when any decimal number is involved. A decimal number is a number that has a decimal mark. A number without a decimal mark is an integer. Examples of exact expressions that are not integer numbers are: ½ π, 2, ...

                    Tip

                    Decimal numbers are shown using the Precision setting and are compared using the Tolerance setting. What is understood as a decimal mark depends on defining the Separator symbols in the Input options section.

                    You can convert an exact expression to approximate by making a simple operation with a decimal number, like multiplying by 1.0.

                    Beware that, despite the correct answer being an integer, the student can always answer a decimal number close enough, and it will be graded as correct. If you want to force the student to answer an integer, you better set the Symbolic answer validation option.

                    Also note some CalcMe commands sometimes return an exact number and sometimes an approximate one. For example: atan(), integrate(), solve(),... When programming the algorithm in Define random variables and functions, keep this in mind.

                    Precision

                    Precision is the number of significant figures or decimal places of variables shown after calculations. Precision affects the decimal numbers shown in wordings, feedback, and so. When rounding, the rule used for tie-breaking is half-up.

                    Consider the number 12.345

                    Precision Rounded to significant figures Rounded to decimal places
                    6 12.345 12.345000
                    5 12.345 12.34500
                    4 12.35 12.3450
                    3 12.3 12.345
                    2 12 12.35
                    1 12 12.3

                    Precision is not the allowed deviation between students and correct answers; that is Tolerance. But precision can harm the comparison if set improperly. Precision affects the correct answer, how it's shown to the student, and also the actual value. The student answer will be compared with the correct answer that has been rounded using precision. If the precision number is smaller than the tolerance number, the correct answer will be rounded too much, probably outside the tolerance range. So remember to set the precision number always more significant than the tolerance number.

                    Tolerance

                    The Tolerance number is used to compare the student answer and the Correct answer. Denote by sa the student's response, ca the correct answer, and n the tolerance.

                    Caution

                    The tolerance options only apply to decimal numbers! For instance, if the question's answer is 10, we may set 10.0 as the correct answer so that the validation system takes into account the defined tolerance.

                    Exact answer

                    The student answer must be exactly equal to the correct answer. It corresponds to what we could call zero-tolerance.

                    For instance, if the correct answer is 12.345

                    Students answer Acceptance
                    12.345
                    12.3450
                    12.3450000001
                    123451000

                    Percent error

                    The student answer has an n% error margin with respect to the correct answer. An answer will be marked as correct if it satisfies

                    For instance, if the correct answer is 12.345

                    Percent error Interval of acceptance
                    10 [11.1105,13.5795]
                    1 [12.22155,12.46845]
                    0.1 [12.332655,12.357345]
                    0.01 [12.3437655,12.3462345]

                    Absolute error

                    The student answer has a margin of error of n. An answer will be marked as correct if it satisfies

                    For instance, if the correct answer is 12.345

                    Absolute error Interval of acceptance
                    1 [11.345,13.345]
                    0.1 [12.245,12.445]
                    0.01 [12.335,12.355]
                    0.001 [12.344,12.346]

                    Significant figures

                    Check that the first n significant figures of student and correct answers are identical. An answer will be marked as correct if it satisfies

                    For instance, if the correct answer is 19.586

                    Student Answer
                    Decimal places 20.01 19.6 19.59 19.58
                    1
                    2
                    3
                    4

                    Decimal places

                    Check that the first n decimal places of student and correct answers are identical. An answer will be marked as correct if it satisfies

                    where trunc is the truncation function. For instance, if the correct answer is 19.586

                    Student Answer
                    Decimal places 19.6 19.59 19.587 19.586
                    1
                    2
                    3

                    Notation

                    These notations apply only to decimal numbers. Numbers and expressions without a decimal point are exact, so these notations don't apply to them. You can always convert an exact expression to decimal by multiplying by 1.0 , for instance.

                    Auto

                    Auto is the default notation if you use significant figures in the Precision setting. Auto notation chooses the best format for each value type: most numbers are shown without changes, but very big or small ones are shown in scientific notation. Decimal trailing zeros are removed too.

                    If you use p significant digits as precision, the floating number x will be shown in scientific notation when . Note that this does not apply to integer numbers.

                    Decimal

                    Decimal notation is like Auto but never shows scientific notation. Note that this notation can produce numbers with lots of digits. For example, you will see in the Algorithm, but you will actually get a long number of hundred digits.

                    This is the only option when working with decimal places as precision.

                    Scientific notation

                    Scientific notation always shows numbers in normalized scientific notation, also called exponential notation. The precision number here means significant digits of the mantissa to show.

                    Consider as the correct answer. By default, the only accepted notation is . If you want to allow students to answer in other formats, you must check the Computer scientific notation option in the Input options section. This way 6.023E23 and 6.023e23 is also accepted.

                    Note that the following answers are not accepted:

                    12.3e+2: because there are too many digits before the decimal point

                    1.0 e+2: because there is an extra space in the middle

                    Concrete number of decimals as assertion

                    As with any other assertion, we can choose if the student answer has a concrete number of significant figures or decimal places.

                    Imagine that we want the student to ask with 1 number after the decimal point. Suppose that the correct answer is 21.5, then.

                    Comparison by rounding

                    As you may have noticed, both significant figures and decimal places comparison methods use the truncation function. Conversely, you may be interested in replicating the same behaviour with the rounding function. Below you can find two alternative procedures to do so.

                    Using absolute error tolerance and a concrete number of decimals assertion

                    The first option will require combining two assertions we have previously analyzed: absolute error and a concrete number of decimals. If we want to check that the student's answer exactly has n decimal places, and these are identical to the correct answer first n decimal places after rounding, we need to set the error margin to and define the precision to n decimal places.

                    For instance, if the correct answer is 19.586

                    Student Answer  
                    Decimal places 19.6 19.59 19.58 19.587 19.586
                    1
                    2
                    3
                    4

                    Using a grading function

                    The second option will require some programming in CalcMe through a grading function. If we want to check that the first n decimal places of the student and correct answers are identical after rounding, we need to define the following user-defined routine (for n=2).

                    Tip

                    You can see more about grading functions here.

                    For instance, if the correct answer is 19.586

                    Student Answer  
                    Decimal places 19.6 19.59 19.58 19.587 19.586
                    1
                    2
                    3
                    4

                    Repeating decimals

                    WirisQuizzes understands decimal numbers with repeating decimals. Repeating decimals is a notation used to represent decimal numbers with periodic expansion, rational numbers. There are three symbols available to mark repeating decimals; you can use any of them. The system is smart enough to know that 0.333⏜=0.3⏜, and of course, that 0.9⏜=1.

                    Thus, students can answer numbers with repeating decimals at any time. The system will recognize it like any other decimal number. If the answer fits the Validation criteria (Tolerance, Mathematically equal, ...), it will be graded as Correct.

                    Conversely, teachers can also set for a Correct answer an endless number with repeating decimals. The student can answer any number, with or without repeating decimals, even a fraction, and it can be graded as correct, depending on the Validation criteria.

                    Finally, remark there is no Additional property to validate if a student answer has or has not repeated decimals nor the possibility to generate variables with repeating decimals. But you can use Literally equal to check if the student answer and the correct answer have the same form or if they both have or have not repeated decimals.

                    Was this article helpful?

                    Yes
                    No
                    Give feedback about this article

                    Related Articles

                    • The four basic operations
                    • Constants
                    • Brackets
                    • Comparison by rounding

                    Numbers representation, tolerance and precision

                    Integers and decimal numbers Tip Precision Tolerance Caution Exact answer Percent error Absolute error Significant figures Decimal places Notation Auto Decimal Scientific notation Concrete number of decimals as assertion Comparison by rounding Using absolute error tolerance and a concrete number of decimals assertion Using a grading function Tip Repeating decimals

                    Making people’s STEM work more meaningful

                    MathType

                    • MathType for Office Tools
                    • MathType for Mac
                    • MathType for Microsoft 365
                    • MathType for Google Workspace
                    • MathType for LMS
                    • MathType for XML Editors
                    • Arabic notation
                    • Our products accessibility
                    • MathType is online

                    WirisQuizzes

                    Learning Lemur

                    Solutions for Education

                    • Blackboard Learn
                    • Brightspace by D2L
                    • Canvas
                    • Google Classroom
                    • Moodle
                    • Schoology

                    Solutions for Publishing Houses

                    Solutions for Technical Writers

                    Solutions for Chemistry

                    Integrations

                    • HTML Editors
                    • MathType in WordPress

                    Pricing

                    Company

                    Careers

                    Blog

                    Contact Us

                    Buy Now

                    Plugin Downloads

                    © Wiris 2025

                    • Cookie Settings
                    • Cookie Policy
                    • Terms of Use
                    • Privacy Policy / GDPR
                    • Student Data Privacy
                    • Compliance
                    • Powered by Helpjuice
                    Expand