0027146: Create command checkplatform
[occt.git] / dox / dev_guides / tests / tests.md
CommitLineData
ba06f8bb 1 Automated Testing System {#occt_dev_guides__tests}
72b7576f 2======================================
3
e5bd0d98 4@tableofcontents
5
dcc0a33e 6@section testmanual_intro Introduction
72b7576f 7
504a8968 8This document provides OCCT developers and contributors with an overview and practical guidelines for work with OCCT automatic testing system.
9
6d368502 10Reading the Introduction should be sufficient for developers to use the test system to control non-regression of the modifications they implement in OCCT. Other sections provide a more in-depth description of the test system, required for modifying the tests and adding new test cases.
72b7576f 11
dcc0a33e 12@subsection testmanual_intro_basic Basic Information
72b7576f 13
9d99d3c1 14OCCT automatic testing system is organized around @ref occt_user_guides__test_harness "DRAW Test Harness", a console application based on Tcl (a scripting language) interpreter extended by OCCT-related commands.
e5bd0d98 15
504a8968 16Standard OCCT tests are included with OCCT sources and are located in subdirectory *tests* of the OCCT root folder. Other test folders can be included in the test system, e.g. for testing applications based on OCCT.
e5bd0d98 17
504a8968 18The tests are organized in three levels:
72b7576f 19
504a8968 20 * Group: a group of related test grids, usually testing a particular OCCT functionality (e.g. blend);
21 * Grid: a set of test cases within a group, usually aimed at testing some particular aspect or mode of execution of the relevant functionality (e.g. buildevol);
22 * Test case: a script implementing an individual test (e.g. K4).
23
67d7f07f 24See @ref testmanual_5_1 "Test Groups" chapter for the current list of available test groups and grids.
72b7576f 25
d3013f55 26@note Many tests involve data files (typically CAD models) which are located separately and (except a few) are not included with OCCT code.
27These tests will be skipped if data files are not available.
72b7576f 28
29@subsection testmanual_1_2 Intended Use of Automatic Tests
30
31Each modification made in OCCT code must be checked for non-regression
6d368502 32by running the whole set of tests. The developer who makes the modification
504a8968 33is responsible for running and ensuring non-regression for the tests available to him.
34
6d368502 35Note that many tests are based on data files that are confidential and thus available only at OPEN CASCADE.
36The official certification testing of each change before its integration to master branch of official OCCT Git repository (and finally to the official release) is performed by OPEN CASCADE to ensure non-regression on all existing test cases and supported platforms.
504a8968 37
38Each new non-trivial modification (improvement, bug fix, new feature) in OCCT should be accompanied by a relevant test case suitable for verifying that modification. This test case is to be added by the developer who provides the modification.
72b7576f 39
504a8968 40If a modification affects the result of an existing test case, either the modification should be corrected (if it causes regression) or the affected test cases should be updated to account for the modification.
72b7576f 41
504a8968 42The modifications made in the OCCT code and related test scripts should be included in the same integration to the master branch.
72b7576f 43
44@subsection testmanual_1_3 Quick Start
45
46@subsubsection testmanual_1_3_1 Setup
47
504a8968 48Before running tests, make sure to define environment variable *CSF_TestDataPath* pointing to the directory containing test data files.
504a8968 49
50For this it is recommended to add a file *DrawAppliInit* in the directory which is current at the moment of starting DRAWEXE (normally it is OCCT root directory, <i>$CASROOT </i>). This file is evaluated automatically at the DRAW start.
51
52Example (Windows)
72b7576f 53
6d368502 54~~~~~{.tcl}
504a8968 55set env(CSF_TestDataPath) $env(CSF_TestDataPath)\;d:/occt/test-data
6d368502 56~~~~~
504a8968 57
504a8968 58Note that variable *CSF_TestDataPath* is set to default value at DRAW start, pointing at the folder <i>$CASROOT/data</i>.
87f42a3f 59In this example, subdirectory <i>d:/occt/test-data</i> is added to this path. Similar code could be used on Linux and Mac OS X except that on non-Windows platforms colon ":" should be used as path separator instead of semicolon ";".
72b7576f 60
936f43da 61All tests are run from DRAW command prompt (run *draw.bat* or *draw.sh* to start it).
72b7576f 62
63@subsubsection testmanual_1_3_2 Running Tests
64
504a8968 65To run all tests, type command *testgrid*
72b7576f 66
67Example:
68
504a8968 69~~~~~
70Draw[]> testgrid
71~~~~~
72b7576f 72
ae3eaf7b 73To run only a subset of test cases, give masks for group, grid, and test case names to be executed.
74Each argument is a list of file masks separated with commas or spaces; by default "*" is assumed.
72b7576f 75
504a8968 76Example:
72b7576f 77
504a8968 78~~~~~
87f42a3f 79Draw[]> testgrid bugs caf,moddata*,xde
504a8968 80~~~~~
72b7576f 81
72b7576f 82As the tests progress, the result of each test case is reported.
504a8968 83At the end of the log a summary of test cases is output,
84including the list of detected regressions and improvements, if any.
72b7576f 85
86
504a8968 87Example:
88
72b7576f 89~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.tcl}
90 Tests summary
91
92 CASE 3rdparty export A1: OK
93 ...
94 CASE pipe standard B1: BAD (known problem)
95 CASE pipe standard C1: OK
96 No regressions
97 Total cases: 208 BAD, 31 SKIPPED, 3 IMPROVEMENT, 1791 OK
98 Elapsed time: 1 Hours 14 Minutes 33.7384512019 Seconds
99 Detailed logs are saved in D:/occt/results_2012-06-04T0919
100~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101
dcc0a33e 102The tests are considered as non-regressive if only OK, BAD (i.e. known problem), and SKIPPED (i.e. not executed, typically because of lack of a data file) statuses are reported. See @ref testmanual_details_results "Interpretation of test results" for details.
72b7576f 103
936f43da 104The results and detailed logs of the tests are saved by default to a new subdirectory of the subdirectory *results* in the current folder, whose name is generated automatically using the current date and time, prefixed by Git branch name (if Git is available and current sources are managed by Git).
3f812249 105If necessary, a non-default output directory can be specified using option <i> -outdir</i> followed by a path to the directory. This directory should be new or empty; use option <i>-overwrite</i> to allow writing results in the existing non-empty directory.
72b7576f 106
504a8968 107Example:
108~~~~~
87f42a3f 109Draw[]> testgrid -outdir d:/occt/last_results -overwrite
504a8968 110~~~~~
ae3eaf7b 111In the output directory, a cumulative HTML report <i>summary.html</i> provides links to reports on each test case. An additional report in JUnit-style XML format can be output for use in Jenkins or other continuous integration system.
504a8968 112
dcb359e0 113To re-run the test cases, which were detected as regressions on the previous run, option <i>-regress dirname</i> should be used.
114<i>dirname</i> is a path to the directory containing the results of the previous run. Only the test cases with *FAILED* and *IMPROVEMENT* statuses will be tested.
b502ddc1 115
116Example:
117~~~~~
118Draw[]> testgrid -regress d:/occt/last_results
119~~~~~
120
936f43da 121Type <i>help testgrid</i> in DRAW prompt to get help on options supported by *testgrid* command:
72b7576f 122
504a8968 123~~~~~
124Draw[3]> help testgrid
125testgrid: Run all tests, or specified group, or one grid
87f42a3f 126 Use: testgrid [groupmask [gridmask [casemask]]] [options...]
504a8968 127 Allowed options are:
128 -parallel N: run N parallel processes (default is number of CPUs, 0 to disable)
129 -refresh N: save summary logs every N seconds (default 60, minimal 1, 0 to disable)
130 -outdir dirname: set log directory (should be empty or non-existing)
131 -overwrite: force writing logs in existing non-empty directory
132 -xml filename: write XML report for Jenkins (in JUnit-like format)
936f43da 133 -beep: play sound signal at the end of the tests
dcb359e0 134 -regress dirname: re-run only a set of tests that have been detected as regressions on the previous run.
135 Here "dirname" is a path to the directory containing the results of the previous run.
136 Groups, grids, and test cases to be executed can be specified by the list of file
137 masks separated by spaces or commas; default is all (*).
504a8968 138~~~~~
72b7576f 139
504a8968 140@subsubsection testmanual_1_3_3 Running a Single Test
141
87f42a3f 142To run a single test, type command *test* followed by names of group, grid, and test case.
72b7576f 143
144Example:
145
146~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.tcl}
147 Draw[1]> test blend simple A1
148 CASE blend simple A1: OK
149 Draw[2]>
150~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
151
504a8968 152Note that normally an intermediate output of the script is not shown. The detailed log of the test can be obtained after the test execution by running command <i>"dlog get"</i>.
153
154To see intermediate commands and their output during the test execution, add one more argument
155<i>"echo"</i> at the end of the command line. Note that with this option the log is not collected and summary is not produced.
156
ae3eaf7b 157Type <i>help test</i> in DRAW prompt to get help on options supported by *test* command:
936f43da 158
159~~~~~
160Draw[3]> help test
161test: Run specified test case
162 Use: test group grid casename [options...]
163 Allowed options are:
164 -echo: all commands and results are echoed immediately,
165 but log is not saved and summary is not produced
166 It is also possible to use "1" instead of "-echo"
167 If echo is OFF, log is stored in memory and only summary
168 is output (the log can be obtained with command "dlog get")
169 -outfile filename: set log file (should be non-existing),
170 it is possible to save log file in text file or
171 in html file(with snapshot), for that "filename"
172 should have ".html" extension
173 -overwrite: force writing log in existing file
174 -beep: play sound signal at the end of the test
175 -errors: show all lines from the log report that are recognized as errors
176 This key will be ignored if the "-echo" key is already set.
177~~~~~
178
dcc0a33e 179@subsubsection testmanual_intro_quick_create Creating a New Test
504a8968 180
67d7f07f 181The detailed rules of creation of new tests are given in @ref testmanual_3 "Creation and modification of tests" chapter. The following short description covers the most typical situations:
504a8968 182
6d368502 183Use prefix <i>bug</i> followed by Mantis issue ID and, if necessary, additional suffixes, for naming the test script, data files, and DRAW commands specific for this test case.
504a8968 184
6d368502 1851. If the test requires C++ code, add it as new DRAW command(s) in one of files in *QABugs* package.
67d7f07f 1862. Add script(s) for the test case in the subfolder corresponding to the relevant OCCT module of the group *bugs* <i>($CASROOT/tests/bugs)</i>. See @ref testmanual_5_2 "the correspondence map".
504a8968 1873. In the test script:
188 * Load all necessary DRAW modules by command *pload*.
189 * Use command *locate_data_file* to get a path to data files used by test script. (Make sure to have this command not inside catch statement if it is used.)
251a7984 190 * Use DRAW commands to reproduce the tested situation.
191 * Make sure that in case of failure the test produces a message containing word "Error" or other recognized by the test system as error (add new error patterns in file parse.rules if necessary).
192 * If the test case reports error due to an existing problem and the fix is not available, add @ref testmanual_3_6 "TODO" statement for each error to mark it as a known problem. The TODO statements must be specific so as to match the actually generated messages but not all similar errors.
193 * To check expected output which should be obtained as the test result, add @ref testmanual_3_7 "REQUIRED" statement for each line of output to mark it as required.
194 * If the test case produces error messages (contained in parse.rules), which are expected in that test and should not be considered as its failure (e.g. test for *checkshape* command), add REQUIRED statement for each error to mark it as required output.
1954. If the test uses data file(s) that are not yet present in the test database, it is possible to put them to (sub)directory pointed out by *CSF_TestDataPath* variable for running test. The files should be attached to the Mantis issue corresponding to the tested modification.
1965. Check that the test case runs as expected (test for fix: OK with the fix, FAILED without the fix; test for existing problem: BAD), and integrate it to the Git branch created for the issue.
504a8968 197
198Example:
199
200* Added files:
936f43da 201
504a8968 202~~~~~
3f812249 203git status -short
6d368502 204A tests/bugs/heal/data/bug210_a.brep
205A tests/bugs/heal/data/bug210_b.brep
504a8968 206A tests/bugs/heal/bug210_1
207A tests/bugs/heal/bug210_2
208~~~~~
209
210* Test script
211
6d368502 212~~~~~{.tcl}
504a8968 213puts "OCC210 (case 1): Improve FixShape for touching wires"
214
6d368502 215restore [locate_data_file bug210_a.brep] a
504a8968 216
217fixshape result a 0.01 0.01
218checkshape result
219~~~~~
72b7576f 220
221@section testmanual_2 Organization of Test Scripts
222
223@subsection testmanual_2_1 General Layout
224
225Standard OCCT tests are located in subdirectory tests of the OCCT root folder ($CASROOT).
72b7576f 226
504a8968 227Additional test folders can be added to the test system by defining environment variable *CSF_TestScriptsPath*. This should be list of paths separated by semicolons (*;*) on Windows
6d368502 228or colons (*:*) on Linux or Mac. Upon DRAW launch, path to *tests* subfolder of OCCT is added at the end of this variable automatically.
504a8968 229
230Each test folder is expected to contain:
6d368502 231 * Optional file *parse.rules* defining patterns for interpretation of test results, common for all groups in this folder
72b7576f 232 * One or several test group directories.
233
234Each group directory contains:
235
504a8968 236 * File *grids.list* that identifies this test group and defines list of test grids in it.
237 * Test grids (sub-directories), each containing set of scripts for test cases, and optional files *cases.list*, *parse.rules*, *begin* and *end*.
72b7576f 238 * Optional sub-directory data
72b7576f 239
504a8968 240By convention, names of test groups, grids, and cases should contain no spaces and be lower-case.
241The names *begin, end, data, parse.rules, grids.list* and *cases.list* are reserved.
72b7576f 242
504a8968 243General layout of test scripts is shown in Figure 1.
244
245@image html /dev_guides/tests/images/tests_image001.png "Layout of tests folder"
246@image latex /dev_guides/tests/images/tests_image001.png "Layout of tests folder"
72b7576f 247
72b7576f 248
249@subsection testmanual_2_2 Test Groups
250
251@subsubsection testmanual_2_2_1 Group Names
252
504a8968 253The names of directories of test groups containing systematic test grids correspond to the functionality tested by each group.
254
72b7576f 255Example:
256
504a8968 257~~~~~
72b7576f 258 caf
259 mesh
260 offset
504a8968 261~~~~~
72b7576f 262
504a8968 263Test group *bugs* is used to collect the tests coming from bug reports. Group *demo* collects tests of the test system, DRAW, samples, etc.
72b7576f 264
504a8968 265@subsubsection testmanual_2_2_2 File "grids.list"
266
267This test group contains file *grids.list*, which defines an ordered list of grids in this group in the following format:
72b7576f 268
269~~~~~~~~~~~~~~~~~
270001 gridname1
271002 gridname2
272...
273NNN gridnameN
274~~~~~~~~~~~~~~~~~
275
276Example:
277
278~~~~~~~~~~~~~~~~~
279 001 basic
280 002 advanced
281~~~~~~~~~~~~~~~~~
282
504a8968 283@subsubsection testmanual_2_2_3 File "begin"
72b7576f 284
504a8968 285This file is a Tcl script. It is executed before every test in the current group.
72b7576f 286Usually it loads necessary Draw commands, sets common parameters and defines
287additional Tcl functions used in test scripts.
504a8968 288
72b7576f 289Example:
290
291~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.tcl}
292 pload TOPTEST ;# load topological command
293 set cpulimit 300 ;# set maximum time allowed for script execution
294~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
295
504a8968 296@subsubsection testmanual_2_2_4 File "end"
72b7576f 297
504a8968 298This file is a TCL script. It is executed after every test in the current group. Usually it checks the results of script work, makes a snap-shot of the viewer and writes *TEST COMPLETED* to the output.
72b7576f 299
504a8968 300Note: *TEST COMPLETED* string should be present in the output to indicate that the test is finished without crash.
301
67d7f07f 302See @ref testmanual_3 "Creation and modification of tests" chapter for more information.
504a8968 303
304Example:
305~~~~~
72b7576f 306 if { [isdraw result] } {
307 checkshape result
308 } else {
4ee1bdf4 309 puts "Error: The result shape can not be built"
72b7576f 310 }
4ee1bdf4 311 puts "TEST COMPLETED"
504a8968 312~~~~~
313
314@subsubsection testmanual_2_2_5 File "parse.rules"
315
316The test group may contain *parse.rules* file. This file defines patterns used for analysis of the test execution log and deciding the status of the test run.
317
318Each line in the file should specify a status (single word), followed by a regular expression delimited by slashes (*/*) that will be matched against lines in the test output log to check if it corresponds to this status.
319
251a7984 320The regular expressions should follow <a href="http://www.tcl.tk/man/tcl/TclCmd/re_syntax.htm">Tcl syntax</a>, with a special exception that "\b" is considered as word limit (Perl-style), in addition to "\y" used in Tcl.
504a8968 321
322The rest of the line can contain a comment message, which will be added to the test report when this status is detected.
72b7576f 323
72b7576f 324Example:
325
504a8968 326~~~~~
6d368502 327 FAILED /\b[Ee]xception\b/ exception
328 FAILED /\bError\b/ error
72b7576f 329 SKIPPED /Cannot open file for reading/ data file is missing
330 SKIPPED /Could not read file .*, abandon/ data file is missing
504a8968 331~~~~~
72b7576f 332
333Lines starting with a *#* character and blank lines are ignored to allow comments and spacing.
72b7576f 334
dcc0a33e 335See @ref testmanual_details_results "Interpretation of test results" chapter for details.
504a8968 336
337If a line matches several rules, the first one applies. Rules defined in the grid are checked first, then rules in the group, then rules in the test root directory. This allows defining some rules on the grid level with status *IGNORE* to ignore messages that would otherwise be treated as errors due to the group level rules.
338
72b7576f 339Example:
340
341~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.tcl}
e5bd0d98 342 FAILED /\\bFaulty\\b/ bad shape
72b7576f 343 IGNORE /^Error [23]d = [\d.-]+/ debug output of blend command
344 IGNORE /^Tcl Exception: tolerance ang : [\d.-]+/ blend failure
345~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
346
504a8968 347@subsubsection testmanual_2_2_6 Directory "data"
67d7f07f 348The test group may contain subdirectory *data*, where test scripts shared by different test grids can be put. See also @ref testmanual_2_3_5 "Directory data".
504a8968 349
72b7576f 350@subsection testmanual_2_3 Test Grids
351
352@subsubsection testmanual_2_3_1 Grid Names
353
504a8968 354The folder of a test group can have several sub-directories (Grid 1… Grid N) defining test grids.
355Each directory contains a set of related test cases. The name of a directory should correspond to its contents.
72b7576f 356
357Example:
358
504a8968 359~~~~~
72b7576f 360caf
361 basic
362 bugs
363 presentation
504a8968 364~~~~~
72b7576f 365
504a8968 366Here *caf* is the name of the test group and *basic*, *bugs*, *presentation*, etc. are the names of grids.
72b7576f 367
504a8968 368@subsubsection testmanual_2_3_2 File "begin"
369
370This file is a TCL script executed before every test in the current grid.
72b7576f 371
72b7576f 372Usually it sets variables specific for the current grid.
504a8968 373
72b7576f 374Example:
375
376~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.tcl}
377 set command bopfuse ;# command tested in this grid
378~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
379
504a8968 380@subsubsection testmanual_2_3_3 File "end"
381
382This file is a TCL script executed after every test in current grid.
383
384Usually it executes a specific sequence of commands common for all tests in the grid.
72b7576f 385
72b7576f 386Example:
387
388~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.tcl}
936f43da 389 vdump $imagedir/${casename}.png ;# makes a snap-shot of AIS viewer
72b7576f 390~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
391
504a8968 392@subsubsection testmanual_2_3_4 File "cases.list"
72b7576f 393
394The grid directory can contain an optional file cases.list
504a8968 395defining an alternative location of the test cases.
396This file should contain a single line defining the relative path to the collection of test cases.
72b7576f 397
398Example:
399
504a8968 400~~~~~
72b7576f 401../data/simple
504a8968 402~~~~~
403
404This option is used for creation of several grids of tests with the same data files and operations but performed with differing parameters. The common scripts are usually located place in the common
405subdirectory of the test group, <i>data/simple</i> for example.
72b7576f 406
504a8968 407If file *cases.list* exists, the grid directory should not contain any test cases.
72b7576f 408The specific parameters and pre- and post-processing commands
504a8968 409for test execution in this grid should be defined in the files *begin* and *end*.
410
411
412@subsubsection testmanual_2_3_5 Directory "data"
413
414The test grid may contain subdirectory *data*, containing data files used in tests (BREP, IGES, STEP, etc.) of this grid.
72b7576f 415
416@subsection testmanual_2_4 Test Cases
417
504a8968 418The test case is a TCL script, which performs some operations using DRAW commands
419and produces meaningful messages that can be used to check the validity of the result.
420
72b7576f 421Example:
422
423~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.tcl}
424 pcylinder c1 10 20 ;# create first cylinder
425 pcylinder c2 5 20 ;# create second cylinder
426 ttranslate c2 5 0 10 ;# translate second cylinder to x,y,z
427 bsection result c1 c2 ;# create a section of two cylinders
428 checksection result ;# will output error message if result is bad
429~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
430
504a8968 431The test case can have any name (except for the reserved names *begin, end, data, cases.list* and *parse.rules*).
72b7576f 432For systematic grids it is usually a capital English letter followed by a number.
433
434Example:
435
504a8968 436~~~~~
72b7576f 437 A1
438 A2
439 B1
440 B2
504a8968 441~~~~~
72b7576f 442
504a8968 443Such naming facilitates compact representation of tests execution results in tabular format within HTML reports.
72b7576f 444
72b7576f 445
446@section testmanual_3 Creation And Modification Of Tests
447
448This section describes how to add new tests and update existing ones.
449
450@subsection testmanual_3_1 Choosing Group, Grid, and Test Case Name
451
504a8968 452The new tests are usually added in the frame of processing issues in OCCT Mantis tracker.
72b7576f 453Such tests in general should be added to group bugs, in the grid
67d7f07f 454corresponding to the affected OCCT functionality. See @ref testmanual_5_2 "Mapping of OCCT functionality to grid names in group bugs".
504a8968 455
456New grids can be added as necessary to contain tests for the functionality not yet covered by existing test grids.
457The test case name in the bugs group should be prefixed by the ID of the corresponding issue in Mantis (without leading zeroes) with prefix *bug*. It is recommended to add a suffix providing a hint on the tested situation. If more than one test is added for a bug, they should be distinguished by suffixes; either meaningful or just ordinal numbers.
72b7576f 458
459Example:
460
461~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.tcl}
504a8968 462 bug12345_coaxial
463 bug12345_orthogonal_1
464 bug12345_orthogonal_2
72b7576f 465~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
466
504a8968 467If the new test corresponds to a functionality already covered by the existing systematic test grid (e.g. group *mesh* for *BRepMesh* issues), this test can be added (or moved later by OCC team) to that grid.
72b7576f 468
469@subsection testmanual_3_2 Adding Data Files Required for a Test
470
ae3eaf7b 471It is advisable to make self-contained test scripts whenever possible, so as they could be used in the environments where data files are not available. For that simple geometric objects and shapes can be created using DRAW commands in the test script itself.
504a8968 472
ae3eaf7b 473If the test requires a data file, it should be put to the directory listed in environment variable *CSF_TestDataPath*.
936f43da 474Alternatively, it can be put to subdirectory *data* of the test grid.
475It is recommended to prefix the data file with the corresponding issue id prefixed by *bug*, e.g. *bug12345_face1.brep*, to avoid possible conflicts with names of existing data files.
504a8968 476
ae3eaf7b 477Note that when the test is integrated to the master branch, OCC team will move the data file to the data files repository, to keep OCCT sources repository clean from data files.
504a8968 478
ae3eaf7b 479When you prepare a test script, try to minimize the size of involved data model. For instance, if the problem detected on a big shape can be reproduced on a single face extracted from that shape, use only that face in the test.
504a8968 480
481
482@subsection testmanual_3_3 Adding new DRAW commands
483
484If the test cannot be implemented using available DRAW commands, consider the following possibilities:
485* If the existing DRAW command can be extended to enable possibility required for a test in a natural way (e.g. by adding an option to activate a specific mode of the algorithm), this way is recommended. This change should be appropriately documented in a relevant Mantis issue.
486* If the new command is needed to access OCCT functionality not exposed to DRAW previously, and this command can be potentially reused (for other tests), it should be added to the package where similar commands are implemented (use *getsource* DRAW command to get the package name). The name and arguments of the new command should be chosen to keep similarity with the existing commands. This change should be documented in a relevant Mantis issue.
487* Otherwise the new command implementing the actions needed for this particular test should be added in *QABugs* package. The command name should be formed by the Mantis issue ID prefixed by *bug*, e.g. *bug12345*.
488
489Note that a DRAW command is expected to return 0 in case of a normal completion, and 1 (Tcl exception) if it is incorrectly used (e.g. a wrong number of input arguments). Thus if the new command needs to report a test error, this should be done by outputting an appropriate error message rather than by returning a non-zero value.
ae3eaf7b 490File names must be encoded in the script rather than in the DRAW command and passed to the DRAW command as an argument.
504a8968 491
492@subsection testmanual_3_4 Script Implementation
493
494The test should run commands necessary to perform the tested operations, in general assuming a clean DRAW session. The required DRAW modules should be loaded by *pload* command, if it is not done by *begin* script. The messages produced by commands in a standard output should include identifiable messages on the discovered problems if any.
495
496Usually the script represents a set of commands that a person would run interactively to perform the operation and see its results, with additional comments to explain what happens.
497
72b7576f 498Example:
504a8968 499~~~~~
500# Simple test of fusing box and sphere
501box b 10 10 10
502sphere s 5
503bfuse result b s
504checkshape result
505~~~~~
72b7576f 506
504a8968 507Make sure that file *parse.rules* in the grid or group directory contains a regular expression to catch possible messages indicating the failure of the test.
508
509For instance, for catching errors reported by *checkshape* command relevant grids define a rule to recognize its report by the word *Faulty*:
510
511~~~~~
512FAILED /\bFaulty\b/ bad shape
513~~~~~
514
515For the messages generated in the script it is recommended to use the word 'Error' in the error message.
72b7576f 516
72b7576f 517Example:
518
504a8968 519~~~~~
520set expected_length 11
521if { [expr $actual_length - $expected_length] > 0.001 } {
522 puts "Error: The length of the edge should be $expected_length"
523}
524~~~~~
525
526At the end, the test script should output *TEST COMPLETED* string to mark a successful completion of the script. This is often done by the *end* script in the grid.
527
528When the test script requires a data file, use Tcl procedure *locate_data_file* to get a path to it, instead of putting the path explicitly. This will allow easy move of the data file from OCCT sources repository to the data files repository without the need to update the test script.
72b7576f 529
72b7576f 530Example:
531
504a8968 532~~~~~
533stepread [locate_data_file CAROSKI_COUPELLE.step] a *
534~~~~~
535
936f43da 536When the test needs to produce some snapshots or other artefacts, use Tcl variable *imagedir* as the location where such files should be put.
ae3eaf7b 537* Command *testgrid* sets this variable to the subdirectory of the results folder corresponding to the grid.
538* Command *test* by default creates a dedicated temporary directory in the system temporary folder (normally the one specified by environment variable *TempDir*, *TEMP*, or *TMP*) for each execution, and sets *imagedir* to that location.
539
540However if variable *imagedir* is defined on the top level of Tcl interpretor, command *test* will use it instead of creating a new directory.
936f43da 541
542Use Tcl variable *casename* to prefix all files produced by the test.
543This variable is set to the name of the test case.
ae3eaf7b 544
545The test system can recognize an image file (snapshot) and include it in HTML log and differences if its name starts with the name of the test case (use variable *casename*), optionally followed by underscore or dash and arbitrary suffix.
546
936f43da 547The image format (defined by extension) should be *png*.
72b7576f 548
72b7576f 549Example:
504a8968 550~~~~~
936f43da 551xwd $imagedir/${casename}.png
504a8968 552vdisplay result; vfit
936f43da 553vdump $imagedir/${casename}-axo.png
504a8968 554vfront; vfit
936f43da 555vdump $imagedir/${casename}-front.png
504a8968 556~~~~~
72b7576f 557
504a8968 558would produce:
559~~~~~
560A1.png
561A1-axo.png
562A1-front.png
563~~~~~
72b7576f 564
504a8968 565Note that OCCT must be built with FreeImage support to be able to produce usable images.
72b7576f 566
936f43da 567Other Tcl variables defined during the test execution are:
ae3eaf7b 568- *groupname*: name of the test group;
569- *gridname*: name of the test grid;
570- *dirname*: path to the root directory of the current set of test scripts.
936f43da 571
504a8968 572In order to ensure that the test works as expected in different environments, observe the following additional rules:
573* Avoid using external commands such as *grep, rm,* etc., as these commands can be absent on another system (e.g. on Windows); use facilities provided by Tcl instead.
3f812249 574* Do not put call to *locate_data_file* in catch statement -- this can prevent correct interpretation of the missing data file by the test system.
6d368502 575* Do not use commands *decho* and *dlog* in the test script, to avoid interference with use of these commands by the test system.
72b7576f 576
dcc0a33e 577@subsection testmanual_details_results Interpretation of test results
72b7576f 578
504a8968 579The result of the test is evaluated by checking its output against patterns defined in the files *parse.rules* of the grid and group.
72b7576f 580
504a8968 581The OCCT test system recognizes five statuses of the test execution:
582* SKIPPED: reported if a line matching SKIPPED pattern is found (prior to any FAILED pattern). This indicates that the test cannot be run in the current environment; the most typical case is the absence of the required data file.
6d368502 583* FAILED: reported if a line matching pattern with status FAILED is found (unless it is masked by the preceding IGNORE pattern or a TODO or REQUIRED statement), or if message TEST COMPLETED or at least one of REQUIRED patterns is not found. This indicates that the test has produced a bad or unexpected result, and usually means a regression.
584* BAD: reported if the test script output contains one or several TODO statements and the corresponding number of matching lines in the log. This indicates a known problem. The lines matching TODO statements are not checked against other patterns and thus will not cause a FAILED status.
504a8968 585* IMPROVEMENT: reported if the test script output contains a TODO statement for which no corresponding line is found. This is a possible indication of improvement (a known problem has disappeared).
586* OK: reported if none of the above statuses have been assigned. This means that the test has passed without problems.
72b7576f 587
504a8968 588Other statuses can be specified in *parse.rules* files, these will be classified as FAILED.
72b7576f 589
504a8968 590For integration of the change to OCCT repository, all tests should return either OK or BAD status.
591The new test created for an unsolved problem should return BAD. The new test created for a fixed problem should return FAILED without the fix, and OK with the fix.
72b7576f 592
504a8968 593@subsection testmanual_3_6 Marking BAD cases
72b7576f 594
ae3eaf7b 595If the test produces an invalid result at a certain moment then the corresponding bug should be created in the OCCT issue tracker located at http://tracker.dev.opencascade.org, and the problem should be marked as TODO in the test script.
72b7576f 596
504a8968 597The following statement should be added to such a test script:
598~~~~~
599puts "TODO BugNumber ListOfPlatforms: RegularExpression"
600~~~~~
601
602Here:
603* *BugNumber* is the bug ID in the tracker. For example: #12345.
863f782a 604* *ListOfPlatforms* is a list of platforms, at which the bug is reproduced (Linux, Windows, MacOS, or All). Note that the platform name is custom for the OCCT test system; Use procedure *checkplatform* to get the platform name.
72b7576f 605
606Example:
504a8968 607~~~~~
863f782a 608Draw[2]> checkplatform
609Windows
504a8968 610~~~~~
72b7576f 611
ae3eaf7b 612* RegularExpression is a regular expression, which should be matched against the line indicating the problem in the script output.
72b7576f 613
72b7576f 614Example:
504a8968 615~~~~~
616puts "TODO #22622 Mandriva2008: Abort .* an exception was raised"
617~~~~~
72b7576f 618
504a8968 619The parser checks the test output and if an output line matches the *RegularExpression* then it will be assigned a BAD status instead of FAILED.
72b7576f 620
504a8968 621A separate TODO line must be added for each output line matching an error expression to mark the test as BAD. If not all TODO messages are found in the test log, the test will be considered as possible improvement.
622
623To mark the test as BAD for an incomplete case (when the final *TEST COMPLETE* message is missing) the expression *TEST INCOMPLETE* should be used instead of the regular expression.
72b7576f 624
625Example:
626
504a8968 627~~~~~
628puts "TODO OCC22817 All: exception.+There are no suitable edges"
629puts "TODO OCC22817 All: \\*\\* Exception \\*\\*"
630puts "TODO OCC22817 All: TEST INCOMPLETE"
631~~~~~
72b7576f 632
6d368502 633@subsection testmanual_3_7 Marking required output
634
251a7984 635To check the obtained test output matches the expected results considered correct, add REQUIRED statement for each specific message.
636For that, the following statement should be added to the corresponding test script:
6d368502 637
638~~~~~
639puts "REQUIRED ListOfPlatforms: RegularExpression"
640~~~~~
641
642Here *ListOfPlatforms* and *RegularExpression* have the same meaning as in TODO statements described above.
643
251a7984 644The REQUIRED statement can also be used to mask the message that would normally be interpreted as error (according to the rules defined in *parse.rules*) but should not be considered as such within the current test.
6d368502 645
646Example:
647~~~~~
b502ddc1 648puts "REQUIRED Linux: Faulty shapes in variables faulty_1 to faulty_5"
6d368502 649~~~~~
650
651This statement notifies test system that errors reported by *checkshape* command are expected in that test case, and test should be considered as OK if this message appears, despite of presence of general rule stating that 'Faulty' signals failure.
504a8968 652
6d368502 653If output does not contain required statement, test case will be marked as FAILED.
504a8968 654
655@section testmanual_4 Advanced Use
72b7576f 656
657@subsection testmanual_4_1 Running Tests on Older Versions of OCCT
658
ae3eaf7b 659Sometimes it might be necessary to run tests on the previous versions of OCCT (<= 6.5.4) that do not include this test system. This can be done by adding DRAW configuration file *DrawAppliInit* in the directory, which is current by the moment of DRAW start-up, to load test commands and to define the necessary environment.
72b7576f 660
504a8968 661Note: in OCCT 6.5.3, file *DrawAppliInit* already exists in <i>$CASROOT/src/DrawResources</i>, new commands should be added to this file instead of a new one in the current directory.
662
663For example, let us assume that *d:/occt* contains an up-to-date version of OCCT sources with tests, and the test data archive is unpacked to *d:/test-data*):
664
665~~~~~
666set env(CASROOT) d:/occt
667set env(CSF_TestScriptsPath) $env(CASROOT)/tests
668source $env(CASROOT)/src/DrawResources/TestCommands.tcl
669set env(CSF_TestDataPath) $env(CASROOT)/data;d:/test-data
670return
671~~~~~
672
ae3eaf7b 673Note that on older versions of OCCT the tests are run in compatibility mode and thus not all output of the test command can be captured; this can lead to absence of some error messages (can be reported as either a failure or an improvement).
504a8968 674
675@subsection testmanual_4_2 Adding custom tests
676
677You can extend the test system by adding your own tests. For that it is necessary to add paths to the directory where these tests are located, and one or more additional data directories, to the environment variables *CSF_TestScriptsPath* and *CSF_TestDataPath*. The recommended way for doing this is using DRAW configuration file *DrawAppliInit* located in the directory which is current by the moment of DRAW start-up.
678
4ee1bdf4 679Use Tcl command <i>_path_separator</i> to insert a platform-dependent separator to the path list.
504a8968 680
681For example:
682~~~~~
683set env(CSF_TestScriptsPath) \
684 $env(TestScriptsPath)[_path_separator]d:/MyOCCTProject/tests
685set env(CSF_TestDataPath) \
686 d:/occt/test-data[_path_separator]d:/MyOCCTProject/data
687return ;# this is to avoid an echo of the last command above in cout
688~~~~~
689
690@subsection testmanual_4_3 Parallel execution of tests
691
692For better efficiency, on computers with multiple CPUs the tests can be run in parallel mode. This is default behavior for command *testgrid* : the tests are executed in parallel processes (their number is equal to the number of CPUs available on the system). In order to change this behavior, use option parallel followed by the number of processes to be used (1 or 0 to run sequentially).
72b7576f 693
87f42a3f 694Note that the parallel execution is only possible if Tcl extension package *Thread* is installed.
695If this package is not available, *testgrid* command will output a warning message.
72b7576f 696
504a8968 697@subsection testmanual_4_4 Checking non-regression of performance, memory, and visualization
72b7576f 698
504a8968 699Some test results are very dependent on the characteristics of the workstation, where they are performed, and thus cannot be checked by comparison with some predefined values. These results can be checked for non-regression (after a change in OCCT code) by comparing them with the results produced by the version without this change. The most typical case is comparing the result obtained in a branch created for integration of a fix (CR***) with the results obtained on the master branch before that change is made.
700
701OCCT test system provides a dedicated command *testdiff* for comparing CPU time of execution, memory usage, and images produced by the tests.
702
703~~~~~
704testdiff dir1 dir2 [groupname [gridname]] [options...]
705~~~~~
706Here *dir1* and *dir2* are directories containing logs of two test runs.
707
708Possible options are:
3f812249 709* <i>-save \<filename\> </i> -- saves the resulting log in a specified file (<i>$dir1/diff-$dir2.log</i> by default). HTML log is saved with the same name and extension .html;
710* <i>-status {same|ok|all}</i> -- allows filtering compared cases by their status:
711 * *same* -- only cases with same status are compared (default);
712 * *ok* -- only cases with OK status in both logs are compared;
713 * *all* -- results are compared regardless of status;
714* <i>-verbose \<level\> </i> -- defines the scope of output data:
715 * 1 -- outputs only differences;
716 * 2 -- additionally outputs the list of logs and directories present in one of directories only;
717 * 3 -- (by default) additionally outputs progress messages;
72b7576f 718
72b7576f 719Example:
504a8968 720
721~~~~~
722Draw[]> testdiff results-CR12345-2012-10-10T08:00 results-master-2012-10-09T21:20
723~~~~~
724
725@section testmanual_5 APPENDIX
726
727@subsection testmanual_5_1 Test groups
728
729@subsubsection testmanual_5_1_1 3rdparty
730
731This group allows testing the interaction of OCCT and 3rdparty products.
732
733DRAW module: VISUALIZATION.
734
735| Grid | Commands | Functionality |
736| :---- | :----- | :------- |
737| export | vexport | export of images to different formats |
738| fonts | vtrihedron, vcolorscale, vdrawtext | display of fonts |
739
740
741@subsubsection testmanual_5_1_2 blend
742
743This group allows testing blends (fillets) and related operations.
744
745DRAW module: MODELING.
746
747| Grid | Commands | Functionality |
748| :---- | :----- | :------- |
749| simple | blend | fillets on simple shapes |
750| complex | blend | fillets on complex shapes, non-trivial geometry |
751| tolblend_simple | tolblend, blend | |
752| buildevol | buildevol | |
753| tolblend_buildvol | tolblend, buildevol | use of additional command tolblend |
754| bfuseblend | bfuseblend | |
755| encoderegularity | encoderegularity | |
756
757@subsubsection testmanual_5_1_3 boolean
758
759This group allows testing Boolean operations.
760
761DRAW module: MODELING (packages *BOPTest* and *BRepTest*).
762
763Grids names are based on name of the command used, with suffixes:
3f812249 764* <i>_2d</i> -- for tests operating with 2d objects (wires, wires, 3d objects, etc.);
765* <i>_simple</i> -- for tests operating on simple shapes (boxes, cylinders, toruses, etc.);
766* <i>_complex</i> -- for tests dealing with complex shapes.
504a8968 767
768| Grid | Commands | Functionality |
769| :---- | :----- | :------- |
770| bcommon_2d | bcommon | Common operation (old algorithm), 2d |
771| bcommon_complex | bcommon | Common operation (old algorithm), complex shapes |
772| bcommon_simple | bcommon | Common operation (old algorithm), simple shapes |
773| bcut_2d | bcut | Cut operation (old algorithm), 2d |
774| bcut_complex | bcut | Cut operation (old algorithm), complex shapes |
775| bcut_simple | bcut | Cut operation (old algorithm), simple shapes |
776| bcutblend | bcutblend | |
777| bfuse_2d | bfuse | Fuse operation (old algorithm), 2d |
778| bfuse_complex | bfuse | Fuse operation (old algorithm), complex shapes |
779| bfuse_simple | bfuse | Fuse operation (old algorithm), simple shapes |
780| bopcommon_2d | bopcommon | Common operation, 2d |
781| bopcommon_complex | bopcommon | Common operation, complex shapes |
782| bopcommon_simple | bopcommon | Common operation, simple shapes |
783| bopcut_2d | bopcut | Cut operation, 2d |
784| bopcut_complex | bopcut | Cut operation, complex shapes |
785| bopcut_simple | bopcut | Cut operation, simple shapes |
786| bopfuse_2d | bopfuse | Fuse operation, 2d |
787| bopfuse_complex | bopfuse | Fuse operation, complex shapes |
788| bopfuse_simple | bopfuse | Fuse operation, simple shapes |
789| bopsection | bopsection | Section |
790| boptuc_2d | boptuc | |
791| boptuc_complex | boptuc | |
792| boptuc_simple | boptuc | |
793| bsection | bsection | Section (old algorithm) |
794
795@subsubsection testmanual_5_1_4 bugs
796
797This group allows testing cases coming from Mantis issues.
798
799The grids are organized following OCCT module and category set for the issue in the Mantis tracker.
67d7f07f 800See @ref testmanual_5_2 "Mapping of OCCT functionality to grid names in group bugs" chapter for details.
504a8968 801
802@subsubsection testmanual_5_1_5 caf
803
804This group allows testing OCAF functionality.
805
806DRAW module: OCAFKERNEL.
807
808| Grid | Commands | Functionality |
809| :---- | :----- | :------- |
810| basic | | Basic attributes |
811| bugs | | Saving and restoring of document |
812| driver | | OCAF drivers |
813| named_shape | | *TNaming_NamedShape* attribute |
814| presentation | | *AISPresentation* attributes |
815| tree | | Tree construction attributes |
816| xlink | | XLink attributes |
817
818@subsubsection testmanual_5_1_6 chamfer
819
820This group allows testing chamfer operations.
821
822DRAW module: MODELING.
823
824The test grid name is constructed depending on the type of the tested chamfers. Additional suffix <i>_complex</i> is used for test cases involving complex geometry (e.g. intersections of edges forming a chamfer); suffix <i>_sequence</i> is used for grids where chamfers are computed sequentially.
825
826| Grid | Commands | Functionality |
827| :---- | :----- | :------- |
828| equal_dist | | Equal distances from edge |
829| equal_dist_complex | | Equal distances from edge, complex shapes |
830| equal_dist_sequence | | Equal distances from edge, sequential operations |
831| dist_dist | | Two distances from edge |
832| dist_dist_complex | | Two distances from edge, complex shapes |
833| dist_dist_sequence | | Two distances from edge, sequential operations |
834| dist_angle | | Distance from edge and given angle |
835| dist_angle_complex | | Distance from edge and given angle |
836| dist_angle_sequence | | Distance from edge and given angle |
837
838@subsubsection testmanual_5_1_7 demo
839
840This group allows demonstrating how testing cases are created, and testing DRAW commands and the test system as a whole.
841
842| Grid | Commands | Functionality |
843| :---- | :----- | :------- |
844| draw | getsource, restore | Basic DRAW commands |
845| testsystem | | Testing system |
846| samples | | OCCT samples |
847
848
849@subsubsection testmanual_5_1_8 draft
850
851This group allows testing draft operations.
852
853DRAW module: MODELING.
854
855| Grid | Commands | Functionality |
856| :---- | :----- | :------- |
857| Angle | depouille | Drafts with angle (inclined walls) |
858
859
860@subsubsection testmanual_5_1_9 feat
861
862This group allows testing creation of features on a shape.
863
864DRAW module: MODELING (package *BRepTest*).
865
866| Grid | Commands | Functionality |
867| :---- | :----- | :------- |
868| featdprism | | |
869| featlf | | |
870| featprism | | |
871| featrevol | | |
872| featrf | | |
873
874@subsubsection testmanual_5_1_10 heal
875
876This group allows testing the functionality provided by *ShapeHealing* toolkit.
877
878DRAW module: XSDRAW
879
880| Grid | Commands | Functionality |
881| :---- | :----- | :------- |
882| fix_shape | fixshape | Shape healing |
883| fix_gaps | fixwgaps | Fixing gaps between edges on a wire |
884| same_parameter | sameparameter | Fixing non-sameparameter edges |
885| fix_face_size | DT_ApplySeq | Removal of small faces |
886| elementary_to_revolution | DT_ApplySeq | Conversion of elementary surfaces to revolution |
887| direct_faces | directfaces | Correction of axis of elementary surfaces |
888| drop_small_edges | fixsmall | Removal of small edges |
889| split_angle | DT_SplitAngle | Splitting periodic surfaces by angle |
890| split_angle_advanced | DT_SplitAngle | Splitting periodic surfaces by angle |
891| split_angle_standard | DT_SplitAngle | Splitting periodic surfaces by angle |
892| split_closed_faces | DT_ClosedSplit | Splitting of closed faces |
893| surface_to_bspline | DT_ToBspl | Conversion of surfaces to b-splines |
894| surface_to_bezier | DT_ShapeConvert | Conversion of surfaces to bezier |
895| split_continuity | DT_ShapeDivide | Split surfaces by continuity criterion |
896| split_continuity_advanced | DT_ShapeDivide | Split surfaces by continuity criterion |
897| split_continuity_standard | DT_ShapeDivide | Split surfaces by continuity criterion |
898| surface_to_revolution_advanced | DT_ShapeConvertRev | Convert elementary surfaces to revolutions, complex cases |
899| surface_to_revolution_standard | DT_ShapeConvertRev | Convert elementary surfaces to revolutions, simple cases |
900
901@subsubsection testmanual_5_1_11 mesh
902
4ee1bdf4 903This group allows testing shape tessellation (*BRepMesh*) and shading.
504a8968 904
905DRAW modules: MODELING (package *MeshTest*), VISUALIZATION (package *ViewerTest*)
906
907| Grid | Commands | Functionality |
908| :---- | :----- | :------- |
909| advanced_shading | vdisplay | Shading, complex shapes |
910| standard_shading | vdisplay | Shading, simple shapes |
911| advanced_mesh | mesh | Meshing of complex shapes |
912| standard_mesh | mesh | Meshing of simple shapes |
913| advanced_incmesh | incmesh | Meshing of complex shapes |
914| standard_incmesh | incmesh | Meshing of simple shapes |
915| advanced_incmesh_parallel | incmesh | Meshing of complex shapes, parallel mode |
916| standard_incmesh_parallel | incmesh | Meshing of simple shapes, parallel mode |
917
918@subsubsection testmanual_5_1_12 mkface
919
920This group allows testing creation of simple surfaces.
921
922DRAW module: MODELING (package *BRepTest*)
923
924| Grid | Commands | Functionality |
925| :---- | :----- | :------- |
926| after_trim | mkface | |
927| after_offset | mkface | |
928| after_extsurf_and_offset | mkface | |
929| after_extsurf_and_trim | mkface | |
930| after_revsurf_and_offset | mkface | |
931| mkplane | mkplane | |
932
933@subsubsection testmanual_5_1_13 nproject
934
935This group allows testing normal projection of edges and wires onto a face.
936
937DRAW module: MODELING (package *BRepTest*)
938
939| Grid | Commands | Functionality |
940| :---- | :----- | :------- |
941| Base | nproject | |
942
943@subsubsection testmanual_5_1_14 offset
944
945This group allows testing offset functionality for curves and surfaces.
946
947DRAW module: MODELING (package *BRepTest*)
948
949| Grid | Commands | Functionality |
950| :---- | :----- | :------- |
951| compshape | offsetcompshape | Offset of shapes with removal of some faces |
952| faces_type_a | offsetparameter, offsetload, offsetperform | Offset on a subset of faces with a fillet |
953| faces_type_i | offsetparameter, offsetload, offsetperform | Offset on a subset of faces with a sharp edge |
954| shape_type_a | offsetparameter, offsetload, offsetperform | Offset on a whole shape with a fillet |
955| shape_type_i | offsetparameter, offsetload, offsetperform | Offset on a whole shape with a fillet |
956| shape | offsetshape | |
957| wire_closed_outside_0_005, wire_closed_outside_0_025, wire_closed_outside_0_075, wire_closed_inside_0_005, wire_closed_inside_0_025, wire_closed_inside_0_075, wire_unclosed_outside_0_005, wire_unclosed_outside_0_025, wire_unclosed_outside_0_075 | mkoffset | 2d offset of closed and unclosed planar wires with different offset step and directions of offset ( inside / outside ) |
958
959@subsubsection testmanual_5_1_15 pipe
960
961This group allows testing construction of pipes (sweeping of a contour along profile).
962
963DRAW module: MODELING (package *BRepTest*)
964
965| Grid | Commands | Functionality |
966| :---- | :----- | :------- |
967| Standard | pipe | |
968
969@subsubsection testmanual_5_1_16 prism
970
971This group allows testing construction of prisms.
972
973DRAW module: MODELING (package *BRepTest*)
974
975| Grid | Commands | Functionality |
976| :---- | :----- | :------- |
977| seminf | prism | |
978
979@subsubsection testmanual_5_1_17 sewing
980
981This group allows testing sewing of faces by connecting edges.
982
983DRAW module: MODELING (package *BRepTest*)
984
985| Grid | Commands | Functionality |
986| :---- | :----- | :------- |
987| tol_0_01 | sewing | Sewing faces with tolerance 0.01 |
988| tol_1 | sewing | Sewing faces with tolerance 1 |
989| tol_100 | sewing | Sewing faces with tolerance 100 |
990
991@subsubsection testmanual_5_1_18 thrusection
992
993This group allows testing construction of shell or a solid passing through a set of sections in a given sequence (loft).
994
995| Grid | Commands | Functionality |
996| :---- | :----- | :------- |
997| solids | thrusection | Lofting with resulting solid |
998| not_solids | thrusection | Lofting with resulting shell or face |
999
1000@subsubsection testmanual_5_1_19 xcaf
1001
1002This group allows testing extended data exchange packages.
1003
1004| Grid | Commands | Functionality |
1005| :---- | :----- | :------- |
1006| dxc, dxc_add_ACL, dxc_add_CL, igs_to_dxc, igs_add_ACL, brep_to_igs_add_CL, stp_to_dxc, stp_add_ACL, brep_to_stp_add_CL, brep_to_dxc, add_ACL_brep, brep_add_CL | | Subgroups are divided by format of source file, by format of result file and by type of document modification. For example, *brep_to_igs* means that the source shape in brep format was added to the document, which was saved into igs format after that. The postfix *add_CL* means that colors and layers were initialized in the document before saving and the postfix *add_ACL* corresponds to the creation of assembly and initialization of colors and layers in a document before saving. |
1007
1008
1009@subsection testmanual_5_2 Mapping of OCCT functionality to grid names in group *bugs*
1010
1011| OCCT Module / Mantis category | Toolkits | Test grid in group bugs |
1012| :---------- | :--------- | :---------- |
1013| Application Framework | PTKernel, TKPShape, TKCDF, TKLCAF, TKCAF, TKBinL, TKXmlL, TKShapeSchema, TKPLCAF, TKBin, TKXml, TKPCAF, FWOSPlugin, TKStdLSchema, TKStdSchema, TKTObj, TKBinTObj, TKXmlTObj | caf |
1014| Draw | TKDraw, TKTopTest, TKViewerTest, TKXSDRAW, TKDCAF, TKXDEDRAW, TKTObjDRAW, TKQADraw, DRAWEXE, Problems of testing system | draw |
1015| Shape Healing | TKShHealing | heal |
1016| Mesh | TKMesh, TKXMesh | mesh |
1017| Data Exchange | TKIGES | iges |
1018| Data Exchange | TKSTEPBase, TKSTEPAttr, TKSTEP209, TKSTEP | step |
1019| Data Exchange | TKSTL, TKVRML | stlvrml |
1020| Data Exchange | TKXSBase, TKXCAF, TKXCAFSchema, TKXDEIGES, TKXDESTEP, TKXmlXCAF, TKBinXCAF | xde |
6268cc68 1021| Foundation Classes | TKernel, TKMath | fclasses |
504a8968 1022| Modeling_algorithms | TKGeomAlgo, TKTopAlgo, TKPrim, TKBO, TKBool, TKHLR, TKFillet, TKOffset, TKFeat, TKXMesh | modalg |
1023| Modeling Data | TKG2d, TKG3d, TKGeomBase, TKBRep | moddata |
6ce0df1e 1024| Visualization | TKService, TKV2d, TKV3d, TKOpenGl, TKMeshVS, TKNIS | vis |
504a8968 1025
1026
5ae01c85 1027@subsection testmanual_5_3 Recommended approaches to checking test results
504a8968 1028
1029@subsubsection testmanual_5_3_1 Shape validity
1030
1031Run command *checkshape* on the result (or intermediate) shape and make sure that *parse.rules* of the test grid or group reports bad shapes (usually recognized by word "Faulty") as error.
1032
1033Example
1034~~~~~
1035checkshape result
1036~~~~~
1037
5ae01c85 1038To check the number of faults in the shape command *checkfaults* can be used.
1039
1040Use: checkfaults shape source_shape [ref_value=0]
1041
1042The default syntax of *checkfaults* command:
1043~~~~~
1044checkfaults results a_1
1045~~~~~
1046
1047The command will check the number of faults in the source shape (*a_1*) and compare it
1048with number of faults in the resulting shape (*result*). If shape *result* contains
1049more faults, you will get an error:
1050~~~~~
1051checkfaults results a_1
1052Error : Number of faults is 5
1053~~~~~
1054It is possible to set the reference value for comparison (reference value is 4):
1055
1056~~~~~
1057checkfaults results a_1 4
1058~~~~~
1059
1060If number of faults in the resulting shape is unstable, reference value should be set to "-1".
1061As a result command *checkfaults* will return the following error:
1062
1063~~~~~
1064checkfaults results a_1 -1
1065Error : Number of faults is UNSTABLE
1066~~~~~
1067
504a8968 1068@subsubsection testmanual_5_3_2 Shape tolerance
dcb359e0 1069
504a8968 1070The maximal tolerance of sub-shapes of each kind of the resulting shape can be extracted from output of tolerance command as follows:
1071
1072~~~~~
1073set tolerance [tolerance result]
1074regexp { *FACE +: +MAX=([-0-9.+eE]+)} $tolerance dummy max_face
1075regexp { *EDGE +: +MAX=([-0-9.+eE]+)} $tolerance dummy max_edgee
1076regexp { *VERTEX +: +MAX=([-0-9.+eE]+)} $tolerance dummy max_vertex
1077~~~~~
1078
5ae01c85 1079It is possible to use command *checkmaxtol* to check maximal tolerance of shape and compare it with reference value.
1080
fb60057d 1081Use: checkmaxtol shape [options...]
5ae01c85 1082
1083Allowed options are:
dcb359e0 1084 * <i>-ref</i> -- reference value of maximum tolerance;
1085 * <i>-source</i> -- list of shapes to compare with;
1086 * <i>-min_tol</i> -- minimum tolerance for comparison;
1087 * <i>-multi_tol</i> -- tolerance multiplier.
5ae01c85 1088
5ae01c85 1089The default syntax of *checkmaxtol* command for comparison with the reference value:
1090~~~~~
fb60057d 1091checkmaxtol result -ref 0.00001
5ae01c85 1092~~~~~
1093
1094There is an opportunity to compare max tolerance of resulting shape with max tolerance of source shape.
1095In the following example command *checkmaxtol* gets max tolerance among objects *a_1* and *a_2*.
1096Then it chooses the maximum value between founded tolerance and value -min_tol (0.000001)
1097and multiply it on the coefficient -multi_tol (i.e. 2):
1098
1099~~~~~
fb60057d 1100checkmaxtol result -source {a_1 a_2} -min_tol 0.000001 -multi_tol 2
5ae01c85 1101~~~~~
1102
1103If the value of maximum tolerance more than founded tolerance for comparison, the command will return an error.
1104
fb60057d 1105Also, command *checkmaxtol* can be used to get max tolerance of the shape:
1106
1107~~~~~
1108set maxtol [checkmaxtol result]
1109~~~~~
1110
504a8968 1111@subsubsection testmanual_5_3_3 Shape volume, area, or length
1112
1113Use command *vprops, sprops,* or *lprops* to correspondingly measure volume, area, or length of the shape produced by the test. The value can be extracted from the result of the command by *regexp*.
1114
1115Example:
1116~~~~~
1117# check area of shape result with 1% tolerance
1118regexp {Mass +: +([-0-9.+eE]+)} [sprops result] dummy area
1119if { abs($area - $expected) > 0.1 + 0.01 * abs ($area) } {
1120 puts "Error: The area of result shape is $area, while expected $expected"
1121}
1122~~~~~
1123
1124@subsubsection testmanual_5_3_4 Memory leaks
1125
dcb359e0 1126The test system measures the amount of memory used by each test case. Considerable deviations (as well as the overall difference) in comparison with reference results can be reported by command *testdiff* (see @ref testmanual_4_4).
504a8968 1127
dcb359e0 1128To check memory leak on a particular operation, run it in a cycle, measure the memory consumption at each step and compare it with a threshold value.
1129The command *checktrend* (defined in *tests/bugs/begin*) can be used to analyze a sequence of memory measurements and to get a statistically based evaluation of the leak presence.
504a8968 1130
1131Example:
1132~~~~~
1133set listmem {}
1134for {set i 1} {$i < 100} {incr i} {
1135 # run suspect operation
1136
1137 # check memory usage (with tolerance equal to half page size)
1138 lappend listmem [expr [meminfo w] / 1024]
1139 if { [checktrend $listmem 0 256 "Memory leak detected"] } {
1140 puts "No memory leak, $i iterations"
1141 break
1142 }
1143}
1144~~~~~
1145
1146@subsubsection testmanual_5_3_5 Visualization
1147
dcb359e0 1148The following command sequence allows you to take a snapshot of the viewer, give it the name of the test case, and save in the directory indicated by Tcl variable *imagedir*.
504a8968 1149
1150~~~~~
1151vinit
1152vclear
1153vdisplay result
1154vsetdispmode 1
1155vfit
1156vzfit
1157vdump $imagedir/${casename}_shading.png
1158~~~~~
1159
1160This image will be included in the HTML log produced by *testgrid* command and will be checked for non-regression through comparison of images by command *testdiff*.
5ae01c85 1161
5747059b 1162Also it is possible to use command *checkview* to make a snapshot of the viewer.
1163
1164Use: checkview [options...]
1165Allowed options are:
dcb359e0 1166* <i>-display shapename </i> -- displays shape with name *shapename*;
1167* <i>-3d </i> -- displays shape in 3d viewer;
1168* <i>-2d [ v2d / smallview ] </i> - displays shape in 2d viewer (the default viewer is *smallview*);
1169* <i>-path PATH</i> -- sets the location of the saved viewer screenshot;
1170* <i>-vdispmode N</i> -- sets *vdispmode* for 3d viewer (default value is 1)
1171* <i>-screenshot</i> -- makes a screenshot of already created viewer
1172* The procedure can check a property of shape (length, area or volume) and compare it with value *N*:
1173 * <i>-l [N]</i>
1174 * <i>-s [N]</i>
1175 * <i>-v [N]</i>
1176 * If the current property is equal to value *N*, the shape is marked as valid in the procedure.
1177 * If value *N* is not given, the procedure will mark the shape as valid if the current property is non-zero.
1178* <i>-with {a b c}</i> -- displays shapes *a, b* and *c* together with the shape (if the shape is valid)
1179* <i>-otherwise {d e f}</i> -- displays shapes *d, e* and *f* instead of the shape (if the shape is NOT valid)
1180
1181Note that is required to use either option <i> -2d </i> or option <i> -3d</i>.
5747059b 1182
1183Examples:
1184~~~~~
1185checkview -display result -2d -path ${imagedir}/${test_image}.png
1186checkview -display result -3d -path ${imagedir}/${test_image}.png
1187checkview -display result_2d -2d v2d -path ${imagedir}/${test_image}.png
1188~~~~~
dcb359e0 1189
5747059b 1190~~~~~
1191box a 10 10 10
1192box b 5 5 5 10 10 10
1193bcut result b a
1194set result_vertices [explode result v]
1195checkview -display result -2d -with ${result_vertices} -otherwise { a b } -l -path ${imagedir}/${test_image}.png
1196~~~~~
dcb359e0 1197
5747059b 1198~~~~~
1199box a 10 10 10
1200box b 5 5 5 10 10 10
1201bcut result b a
1202vinit
1203vdisplay a b
1204vfit
1205checkview -screenshot -3d -path ${imagedir}/${test_image}.png
1206~~~~~
1207
5ae01c85 1208@subsubsection testmanual_5_3_6 Number of free edges
1209
dcb359e0 1210Procedure *checkfreebounds* compares the number of free edges with a reference value.
5ae01c85 1211
1212Use: checkfreebounds shape ref_value [options...]
1213
1214Allowed options are:
dcb359e0 1215 * <i>-tol N</i> -- used tolerance (default -0.01);
1216 * <i>-type N</i> -- used type, possible values are "closed" and "opened" (default "closed").
5ae01c85 1217
1218~~~~~
1219checkfreebounds result 13
1220~~~~~
1221
dcb359e0 1222Option <i>-tol N</i> defines tolerance for command *freebounds*, which is used within command *checkfreebounds*.
5ae01c85 1223
dcb359e0 1224Option <i>-type N</i> is used to select the type of counted free edges: closed or open.
5ae01c85 1225
dcb359e0 1226If the number of free edges in the resulting shape is unstable, the reference value should be set to "-1".
5ae01c85 1227As a result command *checkfreebounds* will return the following error:
1228
1229~~~~~
1230checkfreebounds result -1
1231Error : Number of free edges is UNSTABLE
1232~~~~~
1233
1234@subsubsection testmanual_5_3_7 Compare numbers
1235
dcb359e0 1236Procedure *checkreal* checks the equality of two reals with a tolerance (relative and absolute).
5ae01c85 1237
1238Use: checkreal name value expected tol_abs tol_rel
1239
1240~~~~~
1241checkreal "Some important value" $value 5 0.0001 0.01
1242~~~~~
1243
1244@subsubsection testmanual_5_3_8 Check number of sub-shapes
1245
dcb359e0 1246Procedure *checknbshapes* compares the number of sub-shapes in "shape" with the given reference data.
5ae01c85 1247
1248Use: checknbshapes shape [options...]
dcb359e0 1249
5ae01c85 1250Allowed options are:
dcb359e0 1251 * <i>-vertex N
5ae01c85 1252 * -edge N
1253 * -wire N
1254 * -face N
1255 * -shell N
1256 * -solid N
1257 * -compsolid N
1258 * -compound N
1259 * -shape N
dcb359e0 1260 * -t</i> -- compares the number of sub-shapes in "shape" counting
5ae01c85 1261 the same sub-shapes with different location as different sub-shapes.
dcb359e0 1262 * <i>-m msg</i> -- prints "msg" in case of error
5ae01c85 1263
1264~~~~~
1265checknbshapes result -vertex 8 -edge 4
1266~~~~~
1267
1268@subsubsection testmanual_5_3_9 Check pixel color
1269
dcb359e0 1270Command *checkcolor* can be used to check pixel color.
5ae01c85 1271
1272Use: checkcolor x y red green blue
1273
dcb359e0 1274where:
1275 * <i>x, y</i> -- pixel coordinates;
1276 * <i>red green blue</i> -- expected pixel color (values from 0 to 1).
5ae01c85 1277
dcb359e0 1278This procedure checks color with tolerance (5x5 area).
5ae01c85 1279
1280Next example will compare color of point with coordinates x=100 y=100 with RGB color R=1 G=0 B=0.
1281If colors are not equal, procedure will check the nearest ones points (5x5 area)
1282~~~~~
1283checkcolor 100 100 1 0 0
1284~~~~~
728ae8f9 1285
1286@subsubsection testmanual_5_3_10 Compute length, area and volume of input shape
1287
dcb359e0 1288Procedure *checkprops* computes length, area and volume of the input shape.
728ae8f9 1289
1290Use: checkprops shapename [options...]
1291
1292Allowed options are:
dcb359e0 1293 * <i>-l LENGTH</i> -- command *lprops*, computes the mass properties of all edges in the shape with a linear density of 1;
1294 * <i>-s AREA</i> -- command *sprops*, computes the mass properties of all faces with a surface density of 1;
1295 * <i>-v VOLUME</i> -- command *vprops*, computes the mass properties of all solids with a density of 1;
1296 * <i>-eps EPSILON</i> -- the epsilon defines relative precision of computation;
1297 * <i>-equal SHAPE</i> -- compares area, volume and length of input shapes. Puts error if they are not equal;
1298 * <i>-notequal SHAPE</i> -- compares area, volume and length of input shapes. Puts error if they are equal.
728ae8f9 1299
dcb359e0 1300Options <i> -l, -s </i> and <i> -v</i> are independent and can be used in any order. Tolerance *epsilon* is the same for all options.
728ae8f9 1301
1302~~~~~
1303checkprops result -s 6265.68
1304checkprops result -s -equal FaceBrep
1305~~~~~
1306
1307@subsubsection testmanual_5_3_11 Parse output dump and compare it with reference values
1308
1309Procedure *checkdump* is used to parse output dump and compare it with reference values.
1310
1311Use: checkdump shapename [options...]
1312
1313Allowed options are:
dcb359e0 1314 * <i>-name NAME</i> -- list of parsing parameters (e.g. Center, Axis, etc.);
1315 * <i>-ref VALUE</i> -- list of reference values for each parameter in *NAME*;
1316 * <i>-eps EPSILON</i> -- the epsilon defines relative precision of computation.
728ae8f9 1317
1318~~~~~
1319checkdump result -name {Center Axis XAxis YAxis Radii} -ref {{-70 0} {-1 -0} {-1 -0} {0 -1} {20 10}} -eps 0.01
1320~~~~~
1321
1322@subsubsection testmanual_5_3_12 Compute length of input curve
1323
dcb359e0 1324Procedure *checklength* computes length of the input curve.
728ae8f9 1325
1326Use: checklength curvename [options...]
1327
1328Allowed options are:
dcb359e0 1329 * <i>-l LENGTH</i> -- command *length*, computes the length of the input curve with precision of computation;
1330 * <i>-eps EPSILON</i> -- the epsilon defines a relative precision of computation;
1331 * <i>-equal CURVE</i> -- compares the length of input curves. Puts error if they are not equal;
1332 * <i>-notequal CURVE</i> -- compares the length of input curves. Puts error if they are equal.
728ae8f9 1333
1334~~~~~
1335checklength cp1 -l 7.278
1336checklength res -l -equal ext_1
1337~~~~~
5d7a0489 1338@subsubsection testmanual_5_3_13 Check maximum deflection, number of triangles and nodes in mesh
1339
dcb359e0 1340Command *checktrinfo* can be used to to check the maximum deflection, as well as the number of nodes and triangles in mesh.
5d7a0489 1341
1342Use: checktrinfo shapename [options...]
1343
1344Allowed options are:
dcb359e0 1345 * <i>-tri [N]</i> -- compares the current number of triangles in *shapename* mesh with the given reference data.
1346 If reference value N is not given and the current number of triangles is equal to 0, procedure *checktrinfo* will print an error.
1347 * <i>-nod [N]</i> -- compares the current number of nodes in *shapename* mesh with the given reference data.
1348 If reference value N is not given and the current number of nodes is equal to 0, procedure *checktrinfo* will print an error.
1349 * <i>-defl [N]</i> -- compares the current value of maximum deflection in *shapename* mesh with the given reference data.
1350 If reference value N is not given and current maximum deflection is equal to 0, procedure *checktrinfo* will print an error.
1351 * <i>-max_defl N</i> -- compares the current value of maximum deflection in *shapename* mesh with the max possible value.
1352 * <i>-tol_abs_tri N</i> -- absolute tolerance for comparison of number of triangles (default value 0).
1353 * <i>-tol_rel_tri N</i> -- relative tolerance for comparison of number of triangles (default value 0).
1354 * <i>-tol_abs_nod N</i> -- absolute tolerance for comparison of number of nodes (default value 0).
1355 * <i>-tol_rel_nod N</i> -- relative tolerance for comparison of number of nodes (default value 0).
1356 * <i>-tol_abs_defl N</i> -- absolute tolerance for deflection comparison (default value 0).
1357 * <i>-tol_rel_defl N</i> -- relative tolerance for deflection comparison (default value 0).
1358 * <i>-ref [trinfo a]</i> -- compares deflection, number of triangles and nodes in *shapename* and *a*.
1359
1360Note that options <i> -tri, -nod </i> and <i> -defl </i> do not work together with option <i> -ref</i>.
5d7a0489 1361
1362Examples:
1363
dcb359e0 1364Comparison with some reference values:
5d7a0489 1365~~~~~
1366checktrinfo result -tri 129 -nod 131 -defl 0.01
1367~~~~~
1368
dcb359e0 1369Comparison with another mesh:
5d7a0489 1370~~~~~
1371checktrinfo result -ref [tringo a]
1372~~~~~
1373
dcb359e0 1374Comparison of deflection with the max possible value:
5d7a0489 1375~~~~~
1376checktrinfo result -max_defl 1
1377~~~~~
1378
dcb359e0 1379Check that the current values are not equal to zero:
5d7a0489 1380~~~~~
1381checktrinfo result -tri -nod -defl
1382~~~~~
1383
dcb359e0 1384Check that the number of triangles and the number of nodes are not equal to some specific values:
5d7a0489 1385~~~~~
1386checktrinfo result -tri !10 -nod !8
1387~~~~~
1388
dcb359e0 1389It is possible to compare current values with reference values with some tolerances.
1390Use options <i>-tol_\* </i> for that.
5d7a0489 1391~~~~~
1392checktrinfo result -defl 1 -tol_abs_defl 0.001
1393~~~~~