Test cases

Test case results

The test results are stored in seperate documents. One document for each subversion number.
Test resultscomment
607
templateTest results template

Vocabulary

Passed version The latest branch and version on which the test is known to pass
Broken version The latest branch and version on which the test is known to fail. n/a when older than passed version.
ID A unqiue ID to refer to a test. The unique numbers are maintained in this file. Note that the same test can be run on different hardware/interface. Each combination yields a unique id.
Test case An atomic entity that describes the operations needed to test a feature or only a part of it. The test case should:
  • be uniquely identifiable
  • define the complete prerequisites of the test (eg: the target, the interface, the initial state of the system)
  • define the input to be applied to the system in order to execute the test
  • define the expected output
  • contain the output resulted by running the test case
  • contain the result of the test (pass/fail)
Test suite A (completable) collection of test cases
Testing Testing refers to running the test suite for a specific revision of the software, for one or many targets, using one or many JTAG interfaces. Testing should be be stored along with all the other records for that specific revision. For releases, the results can be stored along with the binaries
Target = ANY Any target can be used for this test
Interface = ANY Any interface can be used for this test
Target = "reset_config srst_and_trst" Any target which supports the reset_config above

Test cases

Connectivity

ID Target Interface Description Initial state Input Expected output Pass/Fail
CON001 ALL ALL Telnet connection Power on, jtag target attached On console, type
telnet ip port		 Open On-Chip Debugger
>		 PASS/FAIL
CON002 ALL ALL GDB server connection Power on, jtag target attached On GDB console, type
target remote ip:port		 Remote debugging using 10.0.0.73:3333 PASS/FAIL

Reset

ID Target Interface Description Initial state Input Expected output Pass/Fail
RES001 Fill in! Fill in! Reset halt on a blank target Erase all the content of the flash Connect via the telnet interface and type
reset halt		 Reset should return without error and the output should contain
target state: halted
pc = 0		 PASS/FAIL
RES002 Fill in! Fill in! Reset init on a blank target Erase all the content of the flash Connect via the telnet interface and type
reset init		 Reset should return without error and the output should contain
executing reset script 'name_of_the_script'		 PASS/FAIL
RES003 Fill in! Fill in! Reset after a power cycle of the target Reset the target then power cycle the target Connect via the telnet interface and type
reset halt after the power was detected		 Reset should return without error and the output should contain
target state: halted		 PASS/FAIL
RES004 ARM7/9,reset_config srst_and_trst ANY Reset halt on a blank target where reset halt is supported Erase all the content of the flash Connect via the telnet interface and type
reset halt		 Reset should return without error and the output should contain
target state: halted
pc = 0		 PASS/FAIL
RES005 arm926ejs,reset_config srst_and_trst ANY Reset halt on a blank target where reset halt is supported. This target has problems with the reset vector catch being disabled by TRST Erase all the content of the flash Connect via the telnet interface and type
reset halt		 Reset should return without error and the output should contain
target state: halted
pc = 0		 PASS/FAIL

JTAG Speed

ID Target Interface Description Initial state Input Expected output Pass/Fail
RES001 Fill in! Fill in! 16MHz on normal operation Reset init the target according to RES002 Exercise a memory access over the JTAG, for example
mdw 0x0 32		 The command should run without any errors. If any JTAG checking errors happen, the test failed PASS/FAIL

Debugging

ID Target Interface Description Initial state Input Expected output Pass/Fail
DBG001 Fill in! Fill in! Load is working Reset init is working, RAM is accesible, GDB server is started On the console of the OS:
arm-elf-gdb test_ram.elf
(gdb) target remote ip:port
(gdb) load 		Load should return without error, typical output looks like:
Loading section .text, size 0x14c lma 0x0
Start address 0x40, load size 332
Transfer rate: 180 bytes/sec, 332 bytes/write.
 		PASS/FAIL
DBG002 Fill in! Fill in! Software breakpoint Load the test_ram.elf application, use instructions from GDB001 In the GDB console:
(gdb) monitor arm7_9 sw_bkpts enable
software breakpoints enabled
(gdb) break main
Breakpoint 1 at 0xec: file src/main.c, line 71.
(gdb) continue
Continuing. 		The software breakpoint should be reached, a typical output looks like:
target state: halted
target halted in ARM state due to breakpoint, current mode: Supervisor
cpsr: 0x000000d3 pc: 0x000000ec

Breakpoint 1, main () at src/main.c:71
71 DWORD a = 1; 		PASS/FAIL
DBG003 Fill in! Fill in! Single step in a RAM application Load the test_ram.elf application, use instructions from GDB001, break in main using the instructions from GDB002 In GDB, type
(gdb) step		 The next instruction should be reached, typical output:
(gdb) step
target state: halted
target halted in ARM state due to single step, current mode: Abort
cpsr: 0x20000097 pc: 0x000000f0
target state: halted
target halted in ARM state due to single step, current mode: Abort
cpsr: 0x20000097 pc: 0x000000f4
72 DWORD b = 2; 		PASS/FAIL
DBG004 Fill in! Fill in! Software break points are working after a reset Load the test_ram.elf application, use instructions from GDB001, break in main using the instructions from GDB002 In GDB, type
(gdb) monitor reset
(gdb) load
(gdb) continue
 The breakpoint should be reached, typical output:
target state: halted
target halted in ARM state due to breakpoint, current mode: Supervisor
cpsr: 0x000000d3 pc: 0x000000ec

Breakpoint 1, main () at src/main.c:71
71 DWORD a = 1; 		PASS/FAIL
DBG005 Fill in! Fill in! Hardware breakpoint Flash the test_rom.elf application. Make this test after FLA004 has passed Be sure that gdb_memory_map and gdb_flash_program are enabled. In GDB, type
(gdb) monitor reset
(gdb) load
Loading section .text, size 0x194 lma 0x100000
Start address 0x100040, load size 404
Transfer rate: 179 bytes/sec, 404 bytes/write.
(gdb) monitor arm7_9 force_hw_bkpts enable
force hardware breakpoints enabled
(gdb) break main
Breakpoint 1 at 0x100134: file src/main.c, line 69.
(gdb) continue
 		The breakpoint should be reached, typical output:
Continuing.

Breakpoint 1, main () at src/main.c:69
69 DWORD a = 1;
 		PASS/FAIL
DBG006 Fill in! Fill in! Hardware breakpoint is set after a reset Follow the instructions to flash and insert a hardware breakpoint from DBG005 In GDB, type
(gdb) monitor reset
(gdb) monitor reg pc 0x100000
pc (/32): 0x00100000
(gdb) continue 		The breakpoint should be reached, typical output:
Continuing.

Breakpoint 1, main () at src/main.c:69
69 DWORD a = 1;
 		PASS/FAIL
DBG007 Fill in! Fill in! Single step in ROM Flash the test_rom.elf application and set a breakpoint in main, use DBG005. Make this test after FLA004 has passed Be sure that gdb_memory_map and gdb_flash_program are enabled. In GDB, type
(gdb) monitor reset
(gdb) load
Loading section .text, size 0x194 lma 0x100000
Start address 0x100040, load size 404
Transfer rate: 179 bytes/sec, 404 bytes/write.
(gdb) monitor arm7_9 force_hw_bkpts enable
force hardware breakpoints enabled
(gdb) break main
Breakpoint 1 at 0x100134: file src/main.c, line 69.
(gdb) continue
Continuing.

Breakpoint 1, main () at src/main.c:69
69 DWORD a = 1;
(gdb) step 		The breakpoint should be reached, typical output:
target state: halted
target halted in ARM state due to single step, current mode: Supervisor
cpsr: 0x60000013 pc: 0x0010013c
70 DWORD b = 2;
 		PASS/FAIL

RAM access

Note: these tests are not designed to test/debug the target, but to test functionalities!
ID Target Interface Description Initial state Input Expected output Pass/Fail
RAM001 Fill in! Fill in! 32 bit Write/read RAM Reset init is working On the telnet interface
> mww ram_address 0xdeadbeef 16
> mdw ram_address 32 		The commands should execute without error. A clear failure is a memory access exception. The result of running the commands should be a list of 16 locations 32bit long containing 0xdeadbeef.
> mww 0x0 0xdeadbeef 16
> mdw 0x0 32
0x00000000: deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef
0x00000020: deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef
0x00000040: e1a00000 e59fa51c e59f051c e04aa000 00080017 00009388 00009388 00009388
0x00000060: 00009388 0002c2c0 0002c2c0 000094f8 000094f4 00009388 00009388 00009388
 		PASS/FAIL
RAM001 Fill in! Fill in! 16 bit Write/read RAM Reset init is working On the telnet interface
> mwh ram_address 0xbeef 16
> mdh ram_address 32 		The commands should execute without error. A clear failure is a memory access exception. The result of running the commands should be a list of 16 locations 16bit long containing 0xbeef.
> mwh 0x0 0xbeef 16
> mdh 0x0 32
0x00000000: beef beef beef beef beef beef beef beef beef beef beef beef beef beef beef beef
0x00000020: 00e0 0000 021c 0000 0240 0000 026c 0000 0288 0000 0000 0000 0388 0000 0350 0000
> 		PASS/FAIL
RAM003 Fill in! Fill in! 8 bit Write/read RAM Reset init is working On the telnet interface
> mwb ram_address 0xab 16
> mdb ram_address 32 		The commands should execute without error. A clear failure is a memory access exception. The result of running the commands should be a list of 16 locations 8bit long containing 0xab.
> mwh 0x0 0x0 16
> mwb ram_address 0xab 16
> mdb ram_address 32
0x00000000: ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab ab 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 		PASS/FAIL

Flash access

ID Target Interface Description Initial state Input Expected output Pass/Fail
FLA001 Fill in! Fill in! Flash probe Reset init is working On the telnet interface:
> flash probe 0 		The command should execute without error. The output should state the name of the flash and the starting address. An example of output:
flash 'ecosflash' found at 0x01000000 		PASS/FAIL
FLA002 Fill in! Fill in! flash fillw Reset init is working, flash is probed On the telnet interface
> flash fillw 0x1000000 0xdeadbeef 16 		The commands should execute without error. The output looks like:
wrote 64 bytes to 0x01000000 in 11.610000s (0.091516 kb/s)
To verify the contents of the flash:
> mdw 0x1000000 32
0x01000000: deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef
0x01000020: deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef deadbeef
0x01000040: ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
0x01000060: ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff 		PASS/FAIL
FLA003 Fill in! Fill in! Flash erase Reset init is working, flash is probed On the telnet interface
> flash erase_address 0x1000000 0x2000 		The commands should execute without error.
erased address 0x01000000 length 8192 in 4.970000s To check that the flash has been erased, read at different addresses. The result should always be 0xff. > mdw 0x1000000 32
0x01000000: ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
0x01000020: ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
0x01000040: ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff
0x01000060: ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff ffffffff 		PASS/FAIL
FLA004 Fill in! Fill in! Loading to flash from GDB Reset init is working, flash is probed, connectivity to GDB server is working Start GDB using a ROM elf image, eg: arm-elf-gdb test_rom.elf.
(gdb) target remote ip:port
(gdb) monitor reset
(gdb) load
Loading section .text, size 0x194 lma 0x100000
Start address 0x100040, load size 404
Transfer rate: 179 bytes/sec, 404 bytes/write. (gdb) monitor verify_image path_to_elf_file 		The output should look like:
verified 404 bytes in 5.060000s
The failure message is something like:
Verify operation failed address 0x00200000. Was 0x00 instead of 0x18 		PASS/FAIL