0025912: Exception during reading STEP file with GD&T
[occt.git] / dox / dev_guides / tests / tests.md
CommitLineData
ba06f8bb 1 Automated Testing System {#occt_dev_guides__tests}
72b7576f 2======================================
3
e5bd0d98 4@tableofcontents
5
72b7576f 6@section testmanual_1 Introduction
7
504a8968 8This document provides OCCT developers and contributors with an overview and practical guidelines for work with OCCT automatic testing system.
9
10Reading the Introduction is sufficient for OCCT developers to use the test system to control non-regression of the modifications they implement in OCCT. Other sections provide a more in-depth description of the test system, required for modifying the tests and adding new test cases.
72b7576f 11
12@subsection testmanual_1_1 Basic Information
13
ba06f8bb 14OCCT automatic testing system is organized around DRAW Test Harness @ref occt_user_guides__test_harness "DRAW Test Harness", a console application based on Tcl (a scripting language) interpreter extended by OCCT-related commands.
e5bd0d98 15
504a8968 16Standard OCCT tests are included with OCCT sources and are located in subdirectory *tests* of the OCCT root folder. Other test folders can be included in the test system, e.g. for testing applications based on OCCT.
e5bd0d98 17
504a8968 18The tests are organized in three levels:
72b7576f 19
504a8968 20 * Group: a group of related test grids, usually testing a particular OCCT functionality (e.g. blend);
21 * Grid: a set of test cases within a group, usually aimed at testing some particular aspect or mode of execution of the relevant functionality (e.g. buildevol);
22 * Test case: a script implementing an individual test (e.g. K4).
23
24See <a href="#testmanual_5_1">Test Groups</a> for the current list of available test groups and grids.
72b7576f 25
504a8968 26Some tests involve data files (typically CAD models) which are located separately and are not included with OCCT code. The archive with publicly available test data files should be downloaded and installed independently on OCCT sources (from http://dev.opencascade.org).
72b7576f 27
28@subsection testmanual_1_2 Intended Use of Automatic Tests
29
30Each modification made in OCCT code must be checked for non-regression
31by running the whole set of tests. The developer who does the modification
504a8968 32is responsible for running and ensuring non-regression for the tests available to him.
33
34Note that many tests are based on data files that are confidential and thus available only at OPEN CASCADE. Thus official certification testing of the changes before integration to master branch of official OCCT Git repository (and finally to the official release) is performed by OPEN CASCADE in any case.
35
36Each new non-trivial modification (improvement, bug fix, new feature) in OCCT should be accompanied by a relevant test case suitable for verifying that modification. This test case is to be added by the developer who provides the modification.
72b7576f 37
504a8968 38If a modification affects the result of an existing test case, either the modification should be corrected (if it causes regression) or the affected test cases should be updated to account for the modification.
72b7576f 39
504a8968 40The modifications made in the OCCT code and related test scripts should be included in the same integration to the master branch.
72b7576f 41
42@subsection testmanual_1_3 Quick Start
43
44@subsubsection testmanual_1_3_1 Setup
45
504a8968 46Before running tests, make sure to define environment variable *CSF_TestDataPath* pointing to the directory containing test data files.
47(Publicly available data files can be downloaded from http://dev.opencascade.org separately from OCCT code.)
48
49For this it is recommended to add a file *DrawAppliInit* in the directory which is current at the moment of starting DRAWEXE (normally it is OCCT root directory, <i>$CASROOT </i>). This file is evaluated automatically at the DRAW start.
50
51Example (Windows)
72b7576f 52
53~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.tcl}
504a8968 54set env(CSF_TestDataPath) $env(CSF_TestDataPath)\;d:/occt/test-data
72b7576f 55return ;# this is to avoid an echo of the last command above in cout
504a8968 56
72b7576f 57~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
504a8968 58Note that variable *CSF_TestDataPath* is set to default value at DRAW start, pointing at the folder <i>$CASROOT/data</i>.
87f42a3f 59In this example, subdirectory <i>d:/occt/test-data</i> is added to this path. Similar code could be used on Linux and Mac OS X except that on non-Windows platforms colon ":" should be used as path separator instead of semicolon ";".
72b7576f 60
4ee1bdf4 61All tests are run from DRAW command prompt (run *draw.tcl* or *draw.sh* to start it).
72b7576f 62
63@subsubsection testmanual_1_3_2 Running Tests
64
504a8968 65To run all tests, type command *testgrid*
72b7576f 66
67Example:
68
504a8968 69~~~~~
70Draw[]> testgrid
71~~~~~
72b7576f 72
87f42a3f 73For running only a subset of test cases, give masks for group, grid, and test case names to be executed.
74Each argument is a list of comma- or space-separated file masks; by default "*" is assumed.
72b7576f 75
504a8968 76Example:
72b7576f 77
504a8968 78~~~~~
87f42a3f 79Draw[]> testgrid bugs caf,moddata*,xde
504a8968 80~~~~~
72b7576f 81
72b7576f 82
83As the tests progress, the result of each test case is reported.
504a8968 84At the end of the log a summary of test cases is output,
85including the list of detected regressions and improvements, if any.
72b7576f 86
87
504a8968 88Example:
89
72b7576f 90~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.tcl}
91 Tests summary
92
93 CASE 3rdparty export A1: OK
94 ...
95 CASE pipe standard B1: BAD (known problem)
96 CASE pipe standard C1: OK
97 No regressions
98 Total cases: 208 BAD, 31 SKIPPED, 3 IMPROVEMENT, 1791 OK
99 Elapsed time: 1 Hours 14 Minutes 33.7384512019 Seconds
100 Detailed logs are saved in D:/occt/results_2012-06-04T0919
101~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
102
504a8968 103The tests are considered as non-regressive if only OK, BAD (i.e. known problem), and SKIPPED (i.e. not executed, typically because of lack of a data file) statuses are reported. See <a href="#testmanual_3_5">Interpretation of test results</a> for details.
72b7576f 104
504a8968 105The results and detailed logs of the tests are saved by default to a subdirectory of the current folder, whose name is generated automatically using the current date and time, prefixed by word <i>"results_"</i> and Git branch name (if Git is available and current sources are managed by Git).
106If necessary, a non-default output directory can be specified using option <i> –outdir</i> followed by a path to the directory. This directory should be new or empty; use option –overwrite to allow writing results in existing non-empty directory.
72b7576f 107
504a8968 108Example:
109~~~~~
87f42a3f 110Draw[]> testgrid -outdir d:/occt/last_results -overwrite
504a8968 111~~~~~
112In the output directory, a cumulative HTML report summary.html provides links to reports on each test case. An additional report in JUnit-style XML format can be output for use in Jenkins or other continuous integration system.
113
114Type <i>help testgrid</i> in DRAW prompt to get help on options supported by *testgrid* command.
115
116For example:
72b7576f 117
504a8968 118~~~~~
119Draw[3]> help testgrid
120testgrid: Run all tests, or specified group, or one grid
87f42a3f 121 Use: testgrid [groupmask [gridmask [casemask]]] [options...]
504a8968 122 Allowed options are:
123 -parallel N: run N parallel processes (default is number of CPUs, 0 to disable)
124 -refresh N: save summary logs every N seconds (default 60, minimal 1, 0 to disable)
125 -outdir dirname: set log directory (should be empty or non-existing)
126 -overwrite: force writing logs in existing non-empty directory
127 -xml filename: write XML report for Jenkins (in JUnit-like format)
87f42a3f 128 Groups, grids, and test cases to be executed can be specified by list of file
129 masks, separated by spaces or comma; default is all (*).
504a8968 130~~~~~
72b7576f 131
504a8968 132@subsubsection testmanual_1_3_3 Running a Single Test
133
87f42a3f 134To run a single test, type command *test* followed by names of group, grid, and test case.
72b7576f 135
136Example:
137
138~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.tcl}
139 Draw[1]> test blend simple A1
140 CASE blend simple A1: OK
141 Draw[2]>
142~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
143
504a8968 144Note that normally an intermediate output of the script is not shown. The detailed log of the test can be obtained after the test execution by running command <i>"dlog get"</i>.
145
146To see intermediate commands and their output during the test execution, add one more argument
147<i>"echo"</i> at the end of the command line. Note that with this option the log is not collected and summary is not produced.
148
149@subsubsection testmanual_1_3_4 Creating a New Test
150
151The detailed rules of creation of new tests are given in <a href="#testmanual_3">section 3</a>. The following short description covers the most typical situations:
152
87f42a3f 153Use prefix "bug" followed by Mantis issue ID and, if necessary, additional suffixes, for naming the test script and DRAW commands specific for this test case.
504a8968 154
1551. If the test requires C++ code, add it as new DRAW command(s) in one of files in *QABugs* package. Note that this package defines macros *QVERIFY* and *QCOMPARE*, thus code created for QTest or GoogleTest frameworks can be used with minimal modifications.
1562. Add script(s) for the test case in grid (subfolder) corresponding to the relevant OCCT module of the group bugs <i>($CASROOT/tests/bugs)</i>. See <a href="#testmanual_5_2">the correspondence map</a>.
1573. In the test script:
158 * Load all necessary DRAW modules by command *pload*.
159 * Use command *locate_data_file* to get a path to data files used by test script. (Make sure to have this command not inside catch statement if it is used.)
160 * Use DRAW commands to reproduce the situation being tested.
161 * If test case is added to describe existing problem and the fix is not available, add TODO message for each error to mark it as known problem. The TODO statements must be specific so as to match the actually generated messages but not all similar errors.
87f42a3f 162 * Make sure that in case of failure the test produces message containing word "Error" or other recognized by test system as error (see files parse.rules).
504a8968 1634. If the test case uses data file(s) not yet present in the test database, these can be put to subfolder data of the test grid, and integrated to Git along with the test case.
1645. Check that the test case runs as expected (test for fix: OK with the fix, FAILED without the fix; test for existing problem: BAD), and integrate to Git branch created for the issue.
165
166Example:
167
168* Added files:
169~~~~~
170git status –short
171A tests/bugs/heal/data/OCC210a.brep
172A tests/bugs/heal/data/OCC210a.brep
173A tests/bugs/heal/bug210_1
174A tests/bugs/heal/bug210_2
175~~~~~
176
177* Test script
178
179~~~~~
180puts "OCC210 (case 1): Improve FixShape for touching wires"
181
182restore [locate_data_file OCC210a.brep] a
183
184fixshape result a 0.01 0.01
185checkshape result
186~~~~~
72b7576f 187
188@section testmanual_2 Organization of Test Scripts
189
190@subsection testmanual_2_1 General Layout
191
192Standard OCCT tests are located in subdirectory tests of the OCCT root folder ($CASROOT).
72b7576f 193
504a8968 194Additional test folders can be added to the test system by defining environment variable *CSF_TestScriptsPath*. This should be list of paths separated by semicolons (*;*) on Windows
195or colons (*:*) on Linux or Mac. Upon DRAW launch, path to tests sub-folder of OCCT is added at the end of this variable automatically.
196
197Each test folder is expected to contain:
72b7576f 198 * Optional file parse.rules defining patterns for interpretation of test results, common for all groups in this folder
199 * One or several test group directories.
200
201Each group directory contains:
202
504a8968 203 * File *grids.list* that identifies this test group and defines list of test grids in it.
204 * Test grids (sub-directories), each containing set of scripts for test cases, and optional files *cases.list*, *parse.rules*, *begin* and *end*.
72b7576f 205 * Optional sub-directory data
72b7576f 206
504a8968 207By convention, names of test groups, grids, and cases should contain no spaces and be lower-case.
208The names *begin, end, data, parse.rules, grids.list* and *cases.list* are reserved.
72b7576f 209
504a8968 210General layout of test scripts is shown in Figure 1.
211
212@image html /dev_guides/tests/images/tests_image001.png "Layout of tests folder"
213@image latex /dev_guides/tests/images/tests_image001.png "Layout of tests folder"
72b7576f 214
72b7576f 215
216@subsection testmanual_2_2 Test Groups
217
218@subsubsection testmanual_2_2_1 Group Names
219
504a8968 220The names of directories of test groups containing systematic test grids correspond to the functionality tested by each group.
221
72b7576f 222Example:
223
504a8968 224~~~~~
72b7576f 225 caf
226 mesh
227 offset
504a8968 228~~~~~
72b7576f 229
504a8968 230Test group *bugs* is used to collect the tests coming from bug reports. Group *demo* collects tests of the test system, DRAW, samples, etc.
72b7576f 231
504a8968 232@subsubsection testmanual_2_2_2 File "grids.list"
233
234This test group contains file *grids.list*, which defines an ordered list of grids in this group in the following format:
72b7576f 235
236~~~~~~~~~~~~~~~~~
237001 gridname1
238002 gridname2
239...
240NNN gridnameN
241~~~~~~~~~~~~~~~~~
242
243Example:
244
245~~~~~~~~~~~~~~~~~
246 001 basic
247 002 advanced
248~~~~~~~~~~~~~~~~~
249
504a8968 250@subsubsection testmanual_2_2_3 File "begin"
72b7576f 251
504a8968 252This file is a Tcl script. It is executed before every test in the current group.
72b7576f 253Usually it loads necessary Draw commands, sets common parameters and defines
254additional Tcl functions used in test scripts.
504a8968 255
72b7576f 256Example:
257
258~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.tcl}
259 pload TOPTEST ;# load topological command
260 set cpulimit 300 ;# set maximum time allowed for script execution
261~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
262
504a8968 263@subsubsection testmanual_2_2_4 File "end"
72b7576f 264
504a8968 265This file is a TCL script. It is executed after every test in the current group. Usually it checks the results of script work, makes a snap-shot of the viewer and writes *TEST COMPLETED* to the output.
72b7576f 266
504a8968 267Note: *TEST COMPLETED* string should be present in the output to indicate that the test is finished without crash.
268
269See <a href="#testmanual_3">section 3</a> for more information.
270
271Example:
272~~~~~
72b7576f 273 if { [isdraw result] } {
274 checkshape result
275 } else {
4ee1bdf4 276 puts "Error: The result shape can not be built"
72b7576f 277 }
4ee1bdf4 278 puts "TEST COMPLETED"
504a8968 279~~~~~
280
281@subsubsection testmanual_2_2_5 File "parse.rules"
282
283The test group may contain *parse.rules* file. This file defines patterns used for analysis of the test execution log and deciding the status of the test run.
284
285Each line in the file should specify a status (single word), followed by a regular expression delimited by slashes (*/*) that will be matched against lines in the test output log to check if it corresponds to this status.
286
287The regular expressions support a subset of the Perl *re* syntax. See also <a href=http://perldoc.perl.org/perlre.html>Perl regular expressions</a>.
288
289The rest of the line can contain a comment message, which will be added to the test report when this status is detected.
72b7576f 290
72b7576f 291Example:
292
504a8968 293~~~~~
e5bd0d98 294 FAILED /\\b[Ee]xception\\b/ exception
295 FAILED /\\bError\\b/ error
72b7576f 296 SKIPPED /Cannot open file for reading/ data file is missing
297 SKIPPED /Could not read file .*, abandon/ data file is missing
504a8968 298~~~~~
72b7576f 299
300Lines starting with a *#* character and blank lines are ignored to allow comments and spacing.
72b7576f 301
504a8968 302See <a href="#testmanual_3_5">Interpretation of test results</a> chapter for details.
303
304If a line matches several rules, the first one applies. Rules defined in the grid are checked first, then rules in the group, then rules in the test root directory. This allows defining some rules on the grid level with status *IGNORE* to ignore messages that would otherwise be treated as errors due to the group level rules.
305
72b7576f 306Example:
307
308~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.tcl}
e5bd0d98 309 FAILED /\\bFaulty\\b/ bad shape
72b7576f 310 IGNORE /^Error [23]d = [\d.-]+/ debug output of blend command
311 IGNORE /^Tcl Exception: tolerance ang : [\d.-]+/ blend failure
312~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
313
504a8968 314@subsubsection testmanual_2_2_6 Directory "data"
315The test group may contain subdirectory *data*, where test scripts shared by different test grids can be put. See also <a href="#testmanual_2_3_5">Directory *data*</a>.
316
72b7576f 317@subsection testmanual_2_3 Test Grids
318
319@subsubsection testmanual_2_3_1 Grid Names
320
504a8968 321The folder of a test group can have several sub-directories (Grid 1… Grid N) defining test grids.
322Each directory contains a set of related test cases. The name of a directory should correspond to its contents.
72b7576f 323
324Example:
325
504a8968 326~~~~~
72b7576f 327caf
328 basic
329 bugs
330 presentation
504a8968 331~~~~~
72b7576f 332
504a8968 333Here *caf* is the name of the test group and *basic*, *bugs*, *presentation*, etc. are the names of grids.
72b7576f 334
504a8968 335@subsubsection testmanual_2_3_2 File "begin"
336
337This file is a TCL script executed before every test in the current grid.
72b7576f 338
72b7576f 339Usually it sets variables specific for the current grid.
504a8968 340
72b7576f 341Example:
342
343~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.tcl}
344 set command bopfuse ;# command tested in this grid
345~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
346
504a8968 347@subsubsection testmanual_2_3_3 File "end"
348
349This file is a TCL script executed after every test in current grid.
350
351Usually it executes a specific sequence of commands common for all tests in the grid.
72b7576f 352
72b7576f 353Example:
354
355~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.tcl}
356 vdump $logdir/${casename}.gif ;# makes a snap-shot of AIS viewer
357~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
358
504a8968 359@subsubsection testmanual_2_3_4 File "cases.list"
72b7576f 360
361The grid directory can contain an optional file cases.list
504a8968 362defining an alternative location of the test cases.
363This file should contain a single line defining the relative path to the collection of test cases.
72b7576f 364
365Example:
366
504a8968 367~~~~~
72b7576f 368../data/simple
504a8968 369~~~~~
370
371This option is used for creation of several grids of tests with the same data files and operations but performed with differing parameters. The common scripts are usually located place in the common
372subdirectory of the test group, <i>data/simple</i> for example.
72b7576f 373
504a8968 374If file *cases.list* exists, the grid directory should not contain any test cases.
72b7576f 375The specific parameters and pre- and post-processing commands
504a8968 376for test execution in this grid should be defined in the files *begin* and *end*.
377
378
379@subsubsection testmanual_2_3_5 Directory "data"
380
381The test grid may contain subdirectory *data*, containing data files used in tests (BREP, IGES, STEP, etc.) of this grid.
72b7576f 382
383@subsection testmanual_2_4 Test Cases
384
504a8968 385The test case is a TCL script, which performs some operations using DRAW commands
386and produces meaningful messages that can be used to check the validity of the result.
387
72b7576f 388Example:
389
390~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.tcl}
391 pcylinder c1 10 20 ;# create first cylinder
392 pcylinder c2 5 20 ;# create second cylinder
393 ttranslate c2 5 0 10 ;# translate second cylinder to x,y,z
394 bsection result c1 c2 ;# create a section of two cylinders
395 checksection result ;# will output error message if result is bad
396~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
397
504a8968 398The test case can have any name (except for the reserved names *begin, end, data, cases.list* and *parse.rules*).
72b7576f 399For systematic grids it is usually a capital English letter followed by a number.
400
401Example:
402
504a8968 403~~~~~
72b7576f 404 A1
405 A2
406 B1
407 B2
504a8968 408~~~~~
72b7576f 409
504a8968 410Such naming facilitates compact representation of tests execution results in tabular format within HTML reports.
72b7576f 411
72b7576f 412
413@section testmanual_3 Creation And Modification Of Tests
414
415This section describes how to add new tests and update existing ones.
416
417@subsection testmanual_3_1 Choosing Group, Grid, and Test Case Name
418
504a8968 419The new tests are usually added in the frame of processing issues in OCCT Mantis tracker.
72b7576f 420Such tests in general should be added to group bugs, in the grid
504a8968 421corresponding to the affected OCCT functionality. See <a href="#testmanual_5_2">Mapping of OCCT functionality to grid names in group *bugs*</a>.
422
423New grids can be added as necessary to contain tests for the functionality not yet covered by existing test grids.
424The test case name in the bugs group should be prefixed by the ID of the corresponding issue in Mantis (without leading zeroes) with prefix *bug*. It is recommended to add a suffix providing a hint on the tested situation. If more than one test is added for a bug, they should be distinguished by suffixes; either meaningful or just ordinal numbers.
72b7576f 425
426Example:
427
428~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~{.tcl}
504a8968 429 bug12345_coaxial
430 bug12345_orthogonal_1
431 bug12345_orthogonal_2
72b7576f 432~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
433
504a8968 434If the new test corresponds to a functionality already covered by the existing systematic test grid (e.g. group *mesh* for *BRepMesh* issues), this test can be added (or moved later by OCC team) to that grid.
72b7576f 435
436@subsection testmanual_3_2 Adding Data Files Required for a Test
437
504a8968 438It is advisable to make self-contained test scripts whenever possible, so as they could be used in environments where data files are not available. For that simple geometric objects and shapes can be created using DRAW commands in the test script itself.
439
440If the test requires a data file, it should be put to subdirectory *data* of the test grid. It is recommended to prefix the data file with the corresponding issue id prefixed by *bug*, e.g. *bug12345_face1.brep*, to avoid possible conflicts with names of existing data files.
441
442Note that when the test is integrated to the master branch, OCC team will move the data file to data files repository, so as to keep OCCT sources repository clean from data files.
443
444When preparing a test script, try to minimize the size of involved data model. For instance, if the problem detected on a big shape can be reproduced on a single face extracted from that shape, use only that face in the test.
445
446
447@subsection testmanual_3_3 Adding new DRAW commands
448
449If the test cannot be implemented using available DRAW commands, consider the following possibilities:
450* If the existing DRAW command can be extended to enable possibility required for a test in a natural way (e.g. by adding an option to activate a specific mode of the algorithm), this way is recommended. This change should be appropriately documented in a relevant Mantis issue.
451* If the new command is needed to access OCCT functionality not exposed to DRAW previously, and this command can be potentially reused (for other tests), it should be added to the package where similar commands are implemented (use *getsource* DRAW command to get the package name). The name and arguments of the new command should be chosen to keep similarity with the existing commands. This change should be documented in a relevant Mantis issue.
452* Otherwise the new command implementing the actions needed for this particular test should be added in *QABugs* package. The command name should be formed by the Mantis issue ID prefixed by *bug*, e.g. *bug12345*.
453
454Note that a DRAW command is expected to return 0 in case of a normal completion, and 1 (Tcl exception) if it is incorrectly used (e.g. a wrong number of input arguments). Thus if the new command needs to report a test error, this should be done by outputting an appropriate error message rather than by returning a non-zero value.
7a7e8cf1 455Also file names must not be encoded in DRAW command but in script, and passed to DRAW command as argument.
504a8968 456
457@subsection testmanual_3_4 Script Implementation
458
459The test should run commands necessary to perform the tested operations, in general assuming a clean DRAW session. The required DRAW modules should be loaded by *pload* command, if it is not done by *begin* script. The messages produced by commands in a standard output should include identifiable messages on the discovered problems if any.
460
461Usually the script represents a set of commands that a person would run interactively to perform the operation and see its results, with additional comments to explain what happens.
462
72b7576f 463Example:
504a8968 464~~~~~
465# Simple test of fusing box and sphere
466box b 10 10 10
467sphere s 5
468bfuse result b s
469checkshape result
470~~~~~
72b7576f 471
504a8968 472Make sure that file *parse.rules* in the grid or group directory contains a regular expression to catch possible messages indicating the failure of the test.
473
474For instance, for catching errors reported by *checkshape* command relevant grids define a rule to recognize its report by the word *Faulty*:
475
476~~~~~
477FAILED /\bFaulty\b/ bad shape
478~~~~~
479
480For the messages generated in the script it is recommended to use the word 'Error' in the error message.
72b7576f 481
72b7576f 482Example:
483
504a8968 484~~~~~
485set expected_length 11
486if { [expr $actual_length - $expected_length] > 0.001 } {
487 puts "Error: The length of the edge should be $expected_length"
488}
489~~~~~
490
491At the end, the test script should output *TEST COMPLETED* string to mark a successful completion of the script. This is often done by the *end* script in the grid.
492
493When the test script requires a data file, use Tcl procedure *locate_data_file* to get a path to it, instead of putting the path explicitly. This will allow easy move of the data file from OCCT sources repository to the data files repository without the need to update the test script.
72b7576f 494
72b7576f 495Example:
496
504a8968 497~~~~~
498stepread [locate_data_file CAROSKI_COUPELLE.step] a *
499~~~~~
500
501When the test needs to produce some snapshots or other artefacts, use Tcl variable *logdir* as the location where such files should be put. Command *testgrid* sets this variable to the subdirectory of the results folder corresponding to the grid. Command *test* sets it to <i>$CASROOT/tmp</i> unless it is already defined. Use Tcl variable *casename* to prefix all files produced by the test. This variable is set to the name of the test case.
72b7576f 502
72b7576f 503Example:
504a8968 504~~~~~
505xwd $logdir/${casename}.png
506vdisplay result; vfit
507vdump $logdir/${casename}-axo.png
508vfront; vfit
509vdump $logdir/${casename}-front.png
510~~~~~
72b7576f 511
504a8968 512would produce:
513~~~~~
514A1.png
515A1-axo.png
516A1-front.png
517~~~~~
72b7576f 518
504a8968 519Note that OCCT must be built with FreeImage support to be able to produce usable images.
72b7576f 520
504a8968 521In order to ensure that the test works as expected in different environments, observe the following additional rules:
522* Avoid using external commands such as *grep, rm,* etc., as these commands can be absent on another system (e.g. on Windows); use facilities provided by Tcl instead.
523* Do not put call to *locate_data_file* in catch statement – this can prevent correct interpretation of the missing data file by the test system.
72b7576f 524
72b7576f 525
504a8968 526@subsection testmanual_3_5 Interpretation of test results
72b7576f 527
504a8968 528The result of the test is evaluated by checking its output against patterns defined in the files *parse.rules* of the grid and group.
72b7576f 529
504a8968 530The OCCT test system recognizes five statuses of the test execution:
531* SKIPPED: reported if a line matching SKIPPED pattern is found (prior to any FAILED pattern). This indicates that the test cannot be run in the current environment; the most typical case is the absence of the required data file.
532* FAILED: reported if a line matching pattern with status FAILED is found (unless it is masked by the preceding IGNORE pattern or a TODO statement), or if message TEST COMPLETED is not found at the end. This indicates that the test has produced a bad or unexpected result, and usually means a regression.
533* BAD: reported if the test script output contains one or several TODO statements and the corresponding number of matching lines in the log. This indicates a known problem . The lines matching TODO statements are not checked against other patterns and thus will not cause a FAILED status.
534* IMPROVEMENT: reported if the test script output contains a TODO statement for which no corresponding line is found. This is a possible indication of improvement (a known problem has disappeared).
535* OK: reported if none of the above statuses have been assigned. This means that the test has passed without problems.
72b7576f 536
504a8968 537Other statuses can be specified in *parse.rules* files, these will be classified as FAILED.
72b7576f 538
504a8968 539For integration of the change to OCCT repository, all tests should return either OK or BAD status.
540The new test created for an unsolved problem should return BAD. The new test created for a fixed problem should return FAILED without the fix, and OK with the fix.
72b7576f 541
504a8968 542@subsection testmanual_3_6 Marking BAD cases
72b7576f 543
504a8968 544If the test produces an invalid result at a certain moment then corresponding bug should be created in the OCCT issue tracker located at http://tracker.dev.opencascade.org, and the problem should be marked as TODO in the test script.
72b7576f 545
504a8968 546The following statement should be added to such a test script:
547~~~~~
548puts "TODO BugNumber ListOfPlatforms: RegularExpression"
549~~~~~
550
551Here:
552* *BugNumber* is the bug ID in the tracker. For example: #12345.
553* *ListOfPlatforms* is a list of platforms at which the bug is reproduced (e.g. Mandriva2008, Windows or All). Note that the platform name is custom for the OCCT test system; it corresponds to the value of environment variable *os_type* defined in DRAW.
72b7576f 554
555Example:
504a8968 556~~~~~
557Draw[2]> puts $env(os_type)
558windows
559~~~~~
72b7576f 560
504a8968 561* RegularExpression is a regular expression which should be matched against the line indicating the problem in the script output.
72b7576f 562
72b7576f 563Example:
504a8968 564~~~~~
565puts "TODO #22622 Mandriva2008: Abort .* an exception was raised"
566~~~~~
72b7576f 567
504a8968 568The parser checks the test output and if an output line matches the *RegularExpression* then it will be assigned a BAD status instead of FAILED.
72b7576f 569
504a8968 570A separate TODO line must be added for each output line matching an error expression to mark the test as BAD. If not all TODO messages are found in the test log, the test will be considered as possible improvement.
571
572To mark the test as BAD for an incomplete case (when the final *TEST COMPLETE* message is missing) the expression *TEST INCOMPLETE* should be used instead of the regular expression.
72b7576f 573
574Example:
575
504a8968 576~~~~~
577puts "TODO OCC22817 All: exception.+There are no suitable edges"
578puts "TODO OCC22817 All: \\*\\* Exception \\*\\*"
579puts "TODO OCC22817 All: TEST INCOMPLETE"
580~~~~~
72b7576f 581
504a8968 582
583
584@section testmanual_4 Advanced Use
72b7576f 585
586@subsection testmanual_4_1 Running Tests on Older Versions of OCCT
587
504a8968 588Sometimes it might be necessary to run tests on previous versions of OCCT (<= 6.5.4) that do not include this test system. This can be done by adding DRAW configuration file *DrawAppliInit* in the directory which is current by the moment of DRAW start-up, to load test commands and to define necessary environment.
72b7576f 589
504a8968 590Note: in OCCT 6.5.3, file *DrawAppliInit* already exists in <i>$CASROOT/src/DrawResources</i>, new commands should be added to this file instead of a new one in the current directory.
591
592For example, let us assume that *d:/occt* contains an up-to-date version of OCCT sources with tests, and the test data archive is unpacked to *d:/test-data*):
593
594~~~~~
595set env(CASROOT) d:/occt
596set env(CSF_TestScriptsPath) $env(CASROOT)/tests
597source $env(CASROOT)/src/DrawResources/TestCommands.tcl
598set env(CSF_TestDataPath) $env(CASROOT)/data;d:/test-data
599return
600~~~~~
601
602Note that on older versions of OCCT the tests are run in compatibility mode and not all output of the test command can be captured; this can lead to absence of some error messages (can be reported as either a failure or an improvement).
603
604@subsection testmanual_4_2 Adding custom tests
605
606You can extend the test system by adding your own tests. For that it is necessary to add paths to the directory where these tests are located, and one or more additional data directories, to the environment variables *CSF_TestScriptsPath* and *CSF_TestDataPath*. The recommended way for doing this is using DRAW configuration file *DrawAppliInit* located in the directory which is current by the moment of DRAW start-up.
607
4ee1bdf4 608Use Tcl command <i>_path_separator</i> to insert a platform-dependent separator to the path list.
504a8968 609
610For example:
611~~~~~
612set env(CSF_TestScriptsPath) \
613 $env(TestScriptsPath)[_path_separator]d:/MyOCCTProject/tests
614set env(CSF_TestDataPath) \
615 d:/occt/test-data[_path_separator]d:/MyOCCTProject/data
616return ;# this is to avoid an echo of the last command above in cout
617~~~~~
618
619@subsection testmanual_4_3 Parallel execution of tests
620
621For better efficiency, on computers with multiple CPUs the tests can be run in parallel mode. This is default behavior for command *testgrid* : the tests are executed in parallel processes (their number is equal to the number of CPUs available on the system). In order to change this behavior, use option parallel followed by the number of processes to be used (1 or 0 to run sequentially).
72b7576f 622
87f42a3f 623Note that the parallel execution is only possible if Tcl extension package *Thread* is installed.
624If this package is not available, *testgrid* command will output a warning message.
72b7576f 625
504a8968 626@subsection testmanual_4_4 Checking non-regression of performance, memory, and visualization
72b7576f 627
504a8968 628Some test results are very dependent on the characteristics of the workstation, where they are performed, and thus cannot be checked by comparison with some predefined values. These results can be checked for non-regression (after a change in OCCT code) by comparing them with the results produced by the version without this change. The most typical case is comparing the result obtained in a branch created for integration of a fix (CR***) with the results obtained on the master branch before that change is made.
629
630OCCT test system provides a dedicated command *testdiff* for comparing CPU time of execution, memory usage, and images produced by the tests.
631
632~~~~~
633testdiff dir1 dir2 [groupname [gridname]] [options...]
634~~~~~
635Here *dir1* and *dir2* are directories containing logs of two test runs.
636
637Possible options are:
ba06f8bb 638* <i>-save \<filename\> </i> - saves the resulting log in a specified file (<i>$dir1/diff-$dir2.log</i> by default). HTML log is saved with the same name and extension .html;
504a8968 639* <i>-status {same|ok|all}</i> - allows filtering compared cases by their status:
640 * *same* - only cases with same status are compared (default);
641 * *ok* - only cases with OK status in both logs are compared;
642 * *all* - results are compared regardless of status;
ba06f8bb 643* <i>-verbose \<level\> </i> - defines the scope of output data:
504a8968 644 * 1 - outputs only differences;
645 * 2 - additionally outputs the list of logs and directories present in one of directories only;
646 * 3 - (by default) additionally outputs progress messages;
72b7576f 647
72b7576f 648Example:
504a8968 649
650~~~~~
651Draw[]> testdiff results-CR12345-2012-10-10T08:00 results-master-2012-10-09T21:20
652~~~~~
653
654@section testmanual_5 APPENDIX
655
656@subsection testmanual_5_1 Test groups
657
658@subsubsection testmanual_5_1_1 3rdparty
659
660This group allows testing the interaction of OCCT and 3rdparty products.
661
662DRAW module: VISUALIZATION.
663
664| Grid | Commands | Functionality |
665| :---- | :----- | :------- |
666| export | vexport | export of images to different formats |
667| fonts | vtrihedron, vcolorscale, vdrawtext | display of fonts |
668
669
670@subsubsection testmanual_5_1_2 blend
671
672This group allows testing blends (fillets) and related operations.
673
674DRAW module: MODELING.
675
676| Grid | Commands | Functionality |
677| :---- | :----- | :------- |
678| simple | blend | fillets on simple shapes |
679| complex | blend | fillets on complex shapes, non-trivial geometry |
680| tolblend_simple | tolblend, blend | |
681| buildevol | buildevol | |
682| tolblend_buildvol | tolblend, buildevol | use of additional command tolblend |
683| bfuseblend | bfuseblend | |
684| encoderegularity | encoderegularity | |
685
686@subsubsection testmanual_5_1_3 boolean
687
688This group allows testing Boolean operations.
689
690DRAW module: MODELING (packages *BOPTest* and *BRepTest*).
691
692Grids names are based on name of the command used, with suffixes:
693* <i>_2d</i> – for tests operating with 2d objects (wires, wires, 3d objects, etc.);
694* <i>_simple</i> – for tests operating on simple shapes (boxes, cylinders, toruses, etc.);
695* <i>_complex</i> – for tests dealing with complex shapes.
696
697| Grid | Commands | Functionality |
698| :---- | :----- | :------- |
699| bcommon_2d | bcommon | Common operation (old algorithm), 2d |
700| bcommon_complex | bcommon | Common operation (old algorithm), complex shapes |
701| bcommon_simple | bcommon | Common operation (old algorithm), simple shapes |
702| bcut_2d | bcut | Cut operation (old algorithm), 2d |
703| bcut_complex | bcut | Cut operation (old algorithm), complex shapes |
704| bcut_simple | bcut | Cut operation (old algorithm), simple shapes |
705| bcutblend | bcutblend | |
706| bfuse_2d | bfuse | Fuse operation (old algorithm), 2d |
707| bfuse_complex | bfuse | Fuse operation (old algorithm), complex shapes |
708| bfuse_simple | bfuse | Fuse operation (old algorithm), simple shapes |
709| bopcommon_2d | bopcommon | Common operation, 2d |
710| bopcommon_complex | bopcommon | Common operation, complex shapes |
711| bopcommon_simple | bopcommon | Common operation, simple shapes |
712| bopcut_2d | bopcut | Cut operation, 2d |
713| bopcut_complex | bopcut | Cut operation, complex shapes |
714| bopcut_simple | bopcut | Cut operation, simple shapes |
715| bopfuse_2d | bopfuse | Fuse operation, 2d |
716| bopfuse_complex | bopfuse | Fuse operation, complex shapes |
717| bopfuse_simple | bopfuse | Fuse operation, simple shapes |
718| bopsection | bopsection | Section |
719| boptuc_2d | boptuc | |
720| boptuc_complex | boptuc | |
721| boptuc_simple | boptuc | |
722| bsection | bsection | Section (old algorithm) |
723
724@subsubsection testmanual_5_1_4 bugs
725
726This group allows testing cases coming from Mantis issues.
727
728The grids are organized following OCCT module and category set for the issue in the Mantis tracker.
729See <a href="#testmanual_5_2">Mapping of OCCT functionality to grid names in group *bugs*</a> for details.
730
731@subsubsection testmanual_5_1_5 caf
732
733This group allows testing OCAF functionality.
734
735DRAW module: OCAFKERNEL.
736
737| Grid | Commands | Functionality |
738| :---- | :----- | :------- |
739| basic | | Basic attributes |
740| bugs | | Saving and restoring of document |
741| driver | | OCAF drivers |
742| named_shape | | *TNaming_NamedShape* attribute |
743| presentation | | *AISPresentation* attributes |
744| tree | | Tree construction attributes |
745| xlink | | XLink attributes |
746
747@subsubsection testmanual_5_1_6 chamfer
748
749This group allows testing chamfer operations.
750
751DRAW module: MODELING.
752
753The test grid name is constructed depending on the type of the tested chamfers. Additional suffix <i>_complex</i> is used for test cases involving complex geometry (e.g. intersections of edges forming a chamfer); suffix <i>_sequence</i> is used for grids where chamfers are computed sequentially.
754
755| Grid | Commands | Functionality |
756| :---- | :----- | :------- |
757| equal_dist | | Equal distances from edge |
758| equal_dist_complex | | Equal distances from edge, complex shapes |
759| equal_dist_sequence | | Equal distances from edge, sequential operations |
760| dist_dist | | Two distances from edge |
761| dist_dist_complex | | Two distances from edge, complex shapes |
762| dist_dist_sequence | | Two distances from edge, sequential operations |
763| dist_angle | | Distance from edge and given angle |
764| dist_angle_complex | | Distance from edge and given angle |
765| dist_angle_sequence | | Distance from edge and given angle |
766
767@subsubsection testmanual_5_1_7 demo
768
769This group allows demonstrating how testing cases are created, and testing DRAW commands and the test system as a whole.
770
771| Grid | Commands | Functionality |
772| :---- | :----- | :------- |
773| draw | getsource, restore | Basic DRAW commands |
774| testsystem | | Testing system |
775| samples | | OCCT samples |
776
777
778@subsubsection testmanual_5_1_8 draft
779
780This group allows testing draft operations.
781
782DRAW module: MODELING.
783
784| Grid | Commands | Functionality |
785| :---- | :----- | :------- |
786| Angle | depouille | Drafts with angle (inclined walls) |
787
788
789@subsubsection testmanual_5_1_9 feat
790
791This group allows testing creation of features on a shape.
792
793DRAW module: MODELING (package *BRepTest*).
794
795| Grid | Commands | Functionality |
796| :---- | :----- | :------- |
797| featdprism | | |
798| featlf | | |
799| featprism | | |
800| featrevol | | |
801| featrf | | |
802
803@subsubsection testmanual_5_1_10 heal
804
805This group allows testing the functionality provided by *ShapeHealing* toolkit.
806
807DRAW module: XSDRAW
808
809| Grid | Commands | Functionality |
810| :---- | :----- | :------- |
811| fix_shape | fixshape | Shape healing |
812| fix_gaps | fixwgaps | Fixing gaps between edges on a wire |
813| same_parameter | sameparameter | Fixing non-sameparameter edges |
814| fix_face_size | DT_ApplySeq | Removal of small faces |
815| elementary_to_revolution | DT_ApplySeq | Conversion of elementary surfaces to revolution |
816| direct_faces | directfaces | Correction of axis of elementary surfaces |
817| drop_small_edges | fixsmall | Removal of small edges |
818| split_angle | DT_SplitAngle | Splitting periodic surfaces by angle |
819| split_angle_advanced | DT_SplitAngle | Splitting periodic surfaces by angle |
820| split_angle_standard | DT_SplitAngle | Splitting periodic surfaces by angle |
821| split_closed_faces | DT_ClosedSplit | Splitting of closed faces |
822| surface_to_bspline | DT_ToBspl | Conversion of surfaces to b-splines |
823| surface_to_bezier | DT_ShapeConvert | Conversion of surfaces to bezier |
824| split_continuity | DT_ShapeDivide | Split surfaces by continuity criterion |
825| split_continuity_advanced | DT_ShapeDivide | Split surfaces by continuity criterion |
826| split_continuity_standard | DT_ShapeDivide | Split surfaces by continuity criterion |
827| surface_to_revolution_advanced | DT_ShapeConvertRev | Convert elementary surfaces to revolutions, complex cases |
828| surface_to_revolution_standard | DT_ShapeConvertRev | Convert elementary surfaces to revolutions, simple cases |
829
830@subsubsection testmanual_5_1_11 mesh
831
4ee1bdf4 832This group allows testing shape tessellation (*BRepMesh*) and shading.
504a8968 833
834DRAW modules: MODELING (package *MeshTest*), VISUALIZATION (package *ViewerTest*)
835
836| Grid | Commands | Functionality |
837| :---- | :----- | :------- |
838| advanced_shading | vdisplay | Shading, complex shapes |
839| standard_shading | vdisplay | Shading, simple shapes |
840| advanced_mesh | mesh | Meshing of complex shapes |
841| standard_mesh | mesh | Meshing of simple shapes |
842| advanced_incmesh | incmesh | Meshing of complex shapes |
843| standard_incmesh | incmesh | Meshing of simple shapes |
844| advanced_incmesh_parallel | incmesh | Meshing of complex shapes, parallel mode |
845| standard_incmesh_parallel | incmesh | Meshing of simple shapes, parallel mode |
846
847@subsubsection testmanual_5_1_12 mkface
848
849This group allows testing creation of simple surfaces.
850
851DRAW module: MODELING (package *BRepTest*)
852
853| Grid | Commands | Functionality |
854| :---- | :----- | :------- |
855| after_trim | mkface | |
856| after_offset | mkface | |
857| after_extsurf_and_offset | mkface | |
858| after_extsurf_and_trim | mkface | |
859| after_revsurf_and_offset | mkface | |
860| mkplane | mkplane | |
861
862@subsubsection testmanual_5_1_13 nproject
863
864This group allows testing normal projection of edges and wires onto a face.
865
866DRAW module: MODELING (package *BRepTest*)
867
868| Grid | Commands | Functionality |
869| :---- | :----- | :------- |
870| Base | nproject | |
871
872@subsubsection testmanual_5_1_14 offset
873
874This group allows testing offset functionality for curves and surfaces.
875
876DRAW module: MODELING (package *BRepTest*)
877
878| Grid | Commands | Functionality |
879| :---- | :----- | :------- |
880| compshape | offsetcompshape | Offset of shapes with removal of some faces |
881| faces_type_a | offsetparameter, offsetload, offsetperform | Offset on a subset of faces with a fillet |
882| faces_type_i | offsetparameter, offsetload, offsetperform | Offset on a subset of faces with a sharp edge |
883| shape_type_a | offsetparameter, offsetload, offsetperform | Offset on a whole shape with a fillet |
884| shape_type_i | offsetparameter, offsetload, offsetperform | Offset on a whole shape with a fillet |
885| shape | offsetshape | |
886| wire_closed_outside_0_005, wire_closed_outside_0_025, wire_closed_outside_0_075, wire_closed_inside_0_005, wire_closed_inside_0_025, wire_closed_inside_0_075, wire_unclosed_outside_0_005, wire_unclosed_outside_0_025, wire_unclosed_outside_0_075 | mkoffset | 2d offset of closed and unclosed planar wires with different offset step and directions of offset ( inside / outside ) |
887
888@subsubsection testmanual_5_1_15 pipe
889
890This group allows testing construction of pipes (sweeping of a contour along profile).
891
892DRAW module: MODELING (package *BRepTest*)
893
894| Grid | Commands | Functionality |
895| :---- | :----- | :------- |
896| Standard | pipe | |
897
898@subsubsection testmanual_5_1_16 prism
899
900This group allows testing construction of prisms.
901
902DRAW module: MODELING (package *BRepTest*)
903
904| Grid | Commands | Functionality |
905| :---- | :----- | :------- |
906| seminf | prism | |
907
908@subsubsection testmanual_5_1_17 sewing
909
910This group allows testing sewing of faces by connecting edges.
911
912DRAW module: MODELING (package *BRepTest*)
913
914| Grid | Commands | Functionality |
915| :---- | :----- | :------- |
916| tol_0_01 | sewing | Sewing faces with tolerance 0.01 |
917| tol_1 | sewing | Sewing faces with tolerance 1 |
918| tol_100 | sewing | Sewing faces with tolerance 100 |
919
920@subsubsection testmanual_5_1_18 thrusection
921
922This group allows testing construction of shell or a solid passing through a set of sections in a given sequence (loft).
923
924| Grid | Commands | Functionality |
925| :---- | :----- | :------- |
926| solids | thrusection | Lofting with resulting solid |
927| not_solids | thrusection | Lofting with resulting shell or face |
928
929@subsubsection testmanual_5_1_19 xcaf
930
931This group allows testing extended data exchange packages.
932
933| Grid | Commands | Functionality |
934| :---- | :----- | :------- |
935| dxc, dxc_add_ACL, dxc_add_CL, igs_to_dxc, igs_add_ACL, brep_to_igs_add_CL, stp_to_dxc, stp_add_ACL, brep_to_stp_add_CL, brep_to_dxc, add_ACL_brep, brep_add_CL | | Subgroups are divided by format of source file, by format of result file and by type of document modification. For example, *brep_to_igs* means that the source shape in brep format was added to the document, which was saved into igs format after that. The postfix *add_CL* means that colors and layers were initialized in the document before saving and the postfix *add_ACL* corresponds to the creation of assembly and initialization of colors and layers in a document before saving. |
936
937
938@subsection testmanual_5_2 Mapping of OCCT functionality to grid names in group *bugs*
939
940| OCCT Module / Mantis category | Toolkits | Test grid in group bugs |
941| :---------- | :--------- | :---------- |
942| Application Framework | PTKernel, TKPShape, TKCDF, TKLCAF, TKCAF, TKBinL, TKXmlL, TKShapeSchema, TKPLCAF, TKBin, TKXml, TKPCAF, FWOSPlugin, TKStdLSchema, TKStdSchema, TKTObj, TKBinTObj, TKXmlTObj | caf |
943| Draw | TKDraw, TKTopTest, TKViewerTest, TKXSDRAW, TKDCAF, TKXDEDRAW, TKTObjDRAW, TKQADraw, DRAWEXE, Problems of testing system | draw |
944| Shape Healing | TKShHealing | heal |
945| Mesh | TKMesh, TKXMesh | mesh |
946| Data Exchange | TKIGES | iges |
947| Data Exchange | TKSTEPBase, TKSTEPAttr, TKSTEP209, TKSTEP | step |
948| Data Exchange | TKSTL, TKVRML | stlvrml |
949| Data Exchange | TKXSBase, TKXCAF, TKXCAFSchema, TKXDEIGES, TKXDESTEP, TKXmlXCAF, TKBinXCAF | xde |
6268cc68 950| Foundation Classes | TKernel, TKMath | fclasses |
504a8968 951| Modeling_algorithms | TKGeomAlgo, TKTopAlgo, TKPrim, TKBO, TKBool, TKHLR, TKFillet, TKOffset, TKFeat, TKXMesh | modalg |
952| Modeling Data | TKG2d, TKG3d, TKGeomBase, TKBRep | moddata |
953| Visualization | TKService, TKV2d, TKV3d, TKOpenGl, TKMeshVS, TKNIS, TKVoxel | vis |
954
955
956@subsection testmanual_5_2 Recommended approaches to checking test results
957
958@subsubsection testmanual_5_3_1 Shape validity
959
960Run command *checkshape* on the result (or intermediate) shape and make sure that *parse.rules* of the test grid or group reports bad shapes (usually recognized by word "Faulty") as error.
961
962Example
963~~~~~
964checkshape result
965~~~~~
966
967@subsubsection testmanual_5_3_2 Shape tolerance
968The maximal tolerance of sub-shapes of each kind of the resulting shape can be extracted from output of tolerance command as follows:
969
970~~~~~
971set tolerance [tolerance result]
972regexp { *FACE +: +MAX=([-0-9.+eE]+)} $tolerance dummy max_face
973regexp { *EDGE +: +MAX=([-0-9.+eE]+)} $tolerance dummy max_edgee
974regexp { *VERTEX +: +MAX=([-0-9.+eE]+)} $tolerance dummy max_vertex
975~~~~~
976
977@subsubsection testmanual_5_3_3 Shape volume, area, or length
978
979Use command *vprops, sprops,* or *lprops* to correspondingly measure volume, area, or length of the shape produced by the test. The value can be extracted from the result of the command by *regexp*.
980
981Example:
982~~~~~
983# check area of shape result with 1% tolerance
984regexp {Mass +: +([-0-9.+eE]+)} [sprops result] dummy area
985if { abs($area - $expected) > 0.1 + 0.01 * abs ($area) } {
986 puts "Error: The area of result shape is $area, while expected $expected"
987}
988~~~~~
989
990@subsubsection testmanual_5_3_4 Memory leaks
991
992The test system measures the amount of memory used by each test case, and considerable deviations (as well as overall difference) comparing with reference results will be reported by *testdiff* command.
993
994The typical approach to checking memory leak on a particular operation is to run this operation in cycle measuring memory consumption at each step and comparing it with some threshold value. Note that file begin in group bugs defines command *checktrend* that can be used to analyze a sequence of memory measurements to get statistically based evaluation of the leak presence.
995
996Example:
997~~~~~
998set listmem {}
999for {set i 1} {$i < 100} {incr i} {
1000 # run suspect operation
1001
1002 # check memory usage (with tolerance equal to half page size)
1003 lappend listmem [expr [meminfo w] / 1024]
1004 if { [checktrend $listmem 0 256 "Memory leak detected"] } {
1005 puts "No memory leak, $i iterations"
1006 break
1007 }
1008}
1009~~~~~
1010
1011@subsubsection testmanual_5_3_5 Visualization
1012
1013Take a snapshot of the viewer, give it the name of the test case, and save in the directory indicated by Tcl variable *imagedir*. Note that this variable directs to the *log* directory if command *testgrid* is active, or to *tmp* subdirectory of the current folder if the test is run interactively.
1014
1015~~~~~
1016vinit
1017vclear
1018vdisplay result
1019vsetdispmode 1
1020vfit
1021vzfit
1022vdump $imagedir/${casename}_shading.png
1023~~~~~
1024
1025This image will be included in the HTML log produced by *testgrid* command and will be checked for non-regression through comparison of images by command *testdiff*.