Improved generator tracking

* Successive executions of the same `GENERATE` macro (e.g. because of a for loop) no longer lead to multiple nested generators. * The same line can now contain multiple `GENERATE` macros without issues. Fixes #1913
2025-08-02 21:35:40 +02:00 · 2020-06-01 19:04:23 +02:00
parent 6c6ebe374a
commit 9500ded83b
14 changed files with 263 additions and 24 deletions
--- a/projects/SelfTest/Baselines/compact.sw.approved.txt
+++ b/projects/SelfTest/Baselines/compact.sw.approved.txt
@@ -30,6 +30,12 @@ CmdLine.tests.cpp:<line number>: passed: spec.matches(fakeTestCase(R"(spec {a} c
 CmdLine.tests.cpp:<line number>: passed: spec.matches(fakeTestCase(R"(spec [a] char)")) for: true
 CmdLine.tests.cpp:<line number>: passed: !(spec.matches(fakeTestCase("differs but has similar tag", "[a]"))) for: !false
 CmdLine.tests.cpp:<line number>: passed: spec.matches(fakeTestCase(R"(spec \ char)")) for: true
+Generators.tests.cpp:<line number>: passed: counter < 7 for: 3 < 7
+Generators.tests.cpp:<line number>: passed: counter < 7 for: 6 < 7
+Generators.tests.cpp:<line number>: passed: i != j for: 1 != 3
+Generators.tests.cpp:<line number>: passed: i != j for: 1 != 4
+Generators.tests.cpp:<line number>: passed: i != j for: 2 != 3
+Generators.tests.cpp:<line number>: passed: i != j for: 2 != 4
 Exception.tests.cpp:<line number>: failed: unexpected exception with message: 'answer := 42' with 1 message: 'expected exception'
 Exception.tests.cpp:<line number>: failed: unexpected exception with message: 'answer := 42'; expression was: thisThrows() with 1 message: 'expected exception'
 Exception.tests.cpp:<line number>: passed: thisThrows() with 1 message: 'answer := 42'
--- a/projects/SelfTest/Baselines/console.std.approved.txt
+++ b/projects/SelfTest/Baselines/console.std.approved.txt
@@ -1380,6 +1380,6 @@ due to unexpected exception with message:
  Why would you throw a std::string?

 ===============================================================================
-test cases:  310 |  236 passed |  70 failed |  4 failed as expected
-assertions: 1701 | 1549 passed | 131 failed | 21 failed as expected
+test cases:  312 |  238 passed |  70 failed |  4 failed as expected
+assertions: 1707 | 1555 passed | 131 failed | 21 failed as expected

--- a/projects/SelfTest/Baselines/console.sw.approved.txt
+++ b/projects/SelfTest/Baselines/console.sw.approved.txt
@@ -243,6 +243,72 @@ CmdLine.tests.cpp:<line number>: PASSED:
 with expansion:
  true

+-------------------------------------------------------------------------------
+#1913 - GENERATE inside a for loop should not keep recreating the generator
+-------------------------------------------------------------------------------
+Generators.tests.cpp:<line number>
+...............................................................................
+
+Generators.tests.cpp:<line number>: PASSED:
+  REQUIRE( counter < 7 )
+with expansion:
+  3 < 7
+
+-------------------------------------------------------------------------------
+#1913 - GENERATE inside a for loop should not keep recreating the generator
+-------------------------------------------------------------------------------
+Generators.tests.cpp:<line number>
+...............................................................................
+
+Generators.tests.cpp:<line number>: PASSED:
+  REQUIRE( counter < 7 )
+with expansion:
+  6 < 7
+
+-------------------------------------------------------------------------------
+#1913 - GENERATEs can share a line
+-------------------------------------------------------------------------------
+Generators.tests.cpp:<line number>
+...............................................................................
+
+Generators.tests.cpp:<line number>: PASSED:
+  REQUIRE( i != j )
+with expansion:
+  1 != 3
+
+-------------------------------------------------------------------------------
+#1913 - GENERATEs can share a line
+-------------------------------------------------------------------------------
+Generators.tests.cpp:<line number>
+...............................................................................
+
+Generators.tests.cpp:<line number>: PASSED:
+  REQUIRE( i != j )
+with expansion:
+  1 != 4
+
+-------------------------------------------------------------------------------
+#1913 - GENERATEs can share a line
+-------------------------------------------------------------------------------
+Generators.tests.cpp:<line number>
+...............................................................................
+
+Generators.tests.cpp:<line number>: PASSED:
+  REQUIRE( i != j )
+with expansion:
+  2 != 3
+
+-------------------------------------------------------------------------------
+#1913 - GENERATEs can share a line
+-------------------------------------------------------------------------------
+Generators.tests.cpp:<line number>
+...............................................................................
+
+Generators.tests.cpp:<line number>: PASSED:
+  REQUIRE( i != j )
+with expansion:
+  2 != 4
+
 -------------------------------------------------------------------------------
 #748 - captures with unexpected exceptions
  outside assertions
@@ -13573,6 +13639,6 @@ Misc.tests.cpp:<line number>
 Misc.tests.cpp:<line number>: PASSED:

 ===============================================================================
-test cases:  310 |  220 passed |  86 failed |  4 failed as expected
-assertions: 1718 | 1549 passed | 148 failed | 21 failed as expected
+test cases:  312 |  222 passed |  86 failed |  4 failed as expected
+assertions: 1724 | 1555 passed | 148 failed | 21 failed as expected

--- a/projects/SelfTest/Baselines/console.swa4.approved.txt
+++ b/projects/SelfTest/Baselines/console.swa4.approved.txt
@@ -243,6 +243,72 @@ CmdLine.tests.cpp:<line number>: PASSED:
 with expansion:
  true

+-------------------------------------------------------------------------------
+#1913 - GENERATE inside a for loop should not keep recreating the generator
+-------------------------------------------------------------------------------
+Generators.tests.cpp:<line number>
+...............................................................................
+
+Generators.tests.cpp:<line number>: PASSED:
+  REQUIRE( counter < 7 )
+with expansion:
+  3 < 7
+
+-------------------------------------------------------------------------------
+#1913 - GENERATE inside a for loop should not keep recreating the generator
+-------------------------------------------------------------------------------
+Generators.tests.cpp:<line number>
+...............................................................................
+
+Generators.tests.cpp:<line number>: PASSED:
+  REQUIRE( counter < 7 )
+with expansion:
+  6 < 7
+
+-------------------------------------------------------------------------------
+#1913 - GENERATEs can share a line
+-------------------------------------------------------------------------------
+Generators.tests.cpp:<line number>
+...............................................................................
+
+Generators.tests.cpp:<line number>: PASSED:
+  REQUIRE( i != j )
+with expansion:
+  1 != 3
+
+-------------------------------------------------------------------------------
+#1913 - GENERATEs can share a line
+-------------------------------------------------------------------------------
+Generators.tests.cpp:<line number>
+...............................................................................
+
+Generators.tests.cpp:<line number>: PASSED:
+  REQUIRE( i != j )
+with expansion:
+  1 != 4
+
+-------------------------------------------------------------------------------
+#1913 - GENERATEs can share a line
+-------------------------------------------------------------------------------
+Generators.tests.cpp:<line number>
+...............................................................................
+
+Generators.tests.cpp:<line number>: PASSED:
+  REQUIRE( i != j )
+with expansion:
+  2 != 3
+
+-------------------------------------------------------------------------------
+#1913 - GENERATEs can share a line
+-------------------------------------------------------------------------------
+Generators.tests.cpp:<line number>
+...............................................................................
+
+Generators.tests.cpp:<line number>: PASSED:
+  REQUIRE( i != j )
+with expansion:
+  2 != 4
+
 -------------------------------------------------------------------------------
 #748 - captures with unexpected exceptions
  outside assertions
@@ -423,6 +489,6 @@ Condition.tests.cpp:<line number>: FAILED:
  CHECK( true != true )

 ===============================================================================
-test cases: 21 | 16 passed | 3 failed | 2 failed as expected
-assertions: 49 | 42 passed | 4 failed | 3 failed as expected
+test cases: 23 | 18 passed | 3 failed | 2 failed as expected
+assertions: 55 | 48 passed | 4 failed | 3 failed as expected

--- a/projects/SelfTest/Baselines/junit.sw.approved.txt
+++ b/projects/SelfTest/Baselines/junit.sw.approved.txt
@@ -1,7 +1,7 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <testsuitesloose text artifact
 >
-  <testsuite name="<exe-name>" errors="17" failures="132" tests="1719" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
+  <testsuite name="<exe-name>" errors="17" failures="132" tests="1725" hostname="tbd" time="{duration}" timestamp="{iso8601-timestamp}">
    <properties>
      <property name="filters" value="~[!nonportable]~[!benchmark]~[approvals]"/>
      <property name="random-seed" value="1"/>
@@ -33,6 +33,8 @@ Nor would this
    <testcase classname="<exe-name>.global" name="#1905 -- test spec parser properly clears internal state between compound tests" time="{duration}" status="run"/>
    <testcase classname="<exe-name>.global" name="#1912 -- test spec parser handles escaping/Various parentheses" time="{duration}" status="run"/>
    <testcase classname="<exe-name>.global" name="#1912 -- test spec parser handles escaping/backslash in test name" time="{duration}" status="run"/>
+    <testcase classname="<exe-name>.global" name="#1913 - GENERATE inside a for loop should not keep recreating the generator" time="{duration}" status="run"/>
+    <testcase classname="<exe-name>.global" name="#1913 - GENERATEs can share a line" time="{duration}" status="run"/>
    <testcase classname="<exe-name>.global" name="#748 - captures with unexpected exceptions/outside assertions" time="{duration}" status="run">
      <error type="TEST_CASE">
 FAILED:
--- a/projects/SelfTest/Baselines/sonarqube.sw.approved.txt
+++ b/projects/SelfTest/Baselines/sonarqube.sw.approved.txt
@@ -875,6 +875,8 @@ Exception.tests.cpp:<line number>
    </testCase>
  </file>
  <file path="projects/<exe-name>/UsageTests/Generators.tests.cpp">
+    <testCase name="#1913 - GENERATE inside a for loop should not keep recreating the generator" duration="{duration}"/>
+    <testCase name="#1913 - GENERATEs can share a line" duration="{duration}"/>
    <testCase name="3x3x3 ints" duration="{duration}"/>
    <testCase name="Copy and then generate a range/from var and iterators" duration="{duration}"/>
    <testCase name="Copy and then generate a range/From a temporary container" duration="{duration}"/>
--- a/projects/SelfTest/Baselines/xml.sw.approved.txt
+++ b/projects/SelfTest/Baselines/xml.sw.approved.txt
@@ -264,6 +264,60 @@ Nor would this
      </Section>
      <OverallResult success="true"/>
    </TestCase>
+    <TestCase name="#1913 - GENERATE inside a for loop should not keep recreating the generator" tags="[generators][regression]" filename="projects/<exe-name>/UsageTests/Generators.tests.cpp" >
+      <Expression success="true" type="REQUIRE" filename="projects/<exe-name>/UsageTests/Generators.tests.cpp" >
+        <Original>
+          counter &lt; 7
+        </Original>
+        <Expanded>
+          3 &lt; 7
+        </Expanded>
+      </Expression>
+      <Expression success="true" type="REQUIRE" filename="projects/<exe-name>/UsageTests/Generators.tests.cpp" >
+        <Original>
+          counter &lt; 7
+        </Original>
+        <Expanded>
+          6 &lt; 7
+        </Expanded>
+      </Expression>
+      <OverallResult success="true"/>
+    </TestCase>
+    <TestCase name="#1913 - GENERATEs can share a line" tags="[generators][regression]" filename="projects/<exe-name>/UsageTests/Generators.tests.cpp" >
+      <Expression success="true" type="REQUIRE" filename="projects/<exe-name>/UsageTests/Generators.tests.cpp" >
+        <Original>
+          i != j
+        </Original>
+        <Expanded>
+          1 != 3
+        </Expanded>
+      </Expression>
+      <Expression success="true" type="REQUIRE" filename="projects/<exe-name>/UsageTests/Generators.tests.cpp" >
+        <Original>
+          i != j
+        </Original>
+        <Expanded>
+          1 != 4
+        </Expanded>
+      </Expression>
+      <Expression success="true" type="REQUIRE" filename="projects/<exe-name>/UsageTests/Generators.tests.cpp" >
+        <Original>
+          i != j
+        </Original>
+        <Expanded>
+          2 != 3
+        </Expanded>
+      </Expression>
+      <Expression success="true" type="REQUIRE" filename="projects/<exe-name>/UsageTests/Generators.tests.cpp" >
+        <Original>
+          i != j
+        </Original>
+        <Expanded>
+          2 != 4
+        </Expanded>
+      </Expression>
+      <OverallResult success="true"/>
+    </TestCase>
    <TestCase name="#748 - captures with unexpected exceptions" tags="[!hide][!shouldfail][!throws][.][failing]" filename="projects/<exe-name>/UsageTests/Exception.tests.cpp" >
      <Section name="outside assertions" filename="projects/<exe-name>/UsageTests/Exception.tests.cpp" >
        <Info>
@@ -16255,7 +16309,7 @@ loose text artifact
      </Section>
      <OverallResult success="true"/>
    </TestCase>
-    <OverallResults successes="1549" failures="149" expectedFailures="21"/>
+    <OverallResults successes="1555" failures="149" expectedFailures="21"/>
  </Group>
-  <OverallResults successes="1549" failures="148" expectedFailures="21"/>
+  <OverallResults successes="1555" failures="148" expectedFailures="21"/>
 </Catch>
--- a/projects/SelfTest/UsageTests/Generators.tests.cpp
+++ b/projects/SelfTest/UsageTests/Generators.tests.cpp
@@ -251,6 +251,22 @@ TEST_CASE("Copy and then generate a range", "[generators]") {
    }
 }

+TEST_CASE("#1913 - GENERATE inside a for loop should not keep recreating the generator", "[regression][generators]") {
+    static int counter = 0;
+    for (int i = 0; i < 3; ++i) {
+        int _ = GENERATE(1, 2);
+        (void)_;
+        ++counter;
+    }
+    // There should be at most 6 (3 * 2) counter increments
+    REQUIRE(counter < 7);
+}
+
+TEST_CASE("#1913 - GENERATEs can share a line", "[regression][generators]") {
+    int i = GENERATE(1, 2); int j = GENERATE(3, 4);
+    REQUIRE(i != j);
+}
+
 #if defined(__clang__)
 #pragma clang diagnostic pop
 #endif