With the latest YAFFS2 (git tag 8a3135f) in a 2.6.32.27 based kernel, I am seeing an out-of-memory crash from operations which do a listing on directories with FIFO files or directories with rapid file deletion/creation.  The issue manifests while doing an ls, find, bash file completion, etc on these types of directories.

 

The root cause seems to be in the making of multiple yaffs_readdir (yaffs_vfs.c) calls to obtain a directory listing where an entry returned in one call changes prior to making a subsequent call.  To test, I created a directory with more listings than are typically returned in a single invocation of yaffs_readdir where an entry returned in the first invocation is a special FIFO file.  Doing an ls on a directory with this structure appears to reliably cause the failure.

bash> mkdir -p test

bash> for i in {0..200} ; do touch test/$i ; done

bash> rm test/125

bash> mkfifo test/125

bash> ls test

 

The ‘ls’ results in the oom killer being invoked (output shown below).  Doing an strace on the ls process shows repeated system calls of getdents(3, /* 186 entries */, 2980) and lseek(3, 130, SEEK_SET).  Apparently the FIFO trips the yaffs_readdir test for “f->f_version != inode->i_version” each time it is invoked, causing the directory listing to remain stuck in an infinite loop until the oom killer is invoked.

 

As noted above, similar behavior occurs when  the directory does not contain a fifo, but instead a second process is rapidly deleting and re-creating file #125.  eg: while [ 1 ] ; do rm test/125 ; date > test/125; done

 

Note that if the FIFO or changing file is the last entry in the directory (eg using filename 201 in the above procedure) then the system behaves normally with no crash or errors.

 

Is this a known issue and/or is there a work-around ?

 

Thank you!

Ian

 

OOM Killer output from ls of test directory with FIFO at the 125th entry.

bash> ls test

ls invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0

Call Trace:

[<ffffffff8010be44>] dump_stack+0x8/0x34

[<ffffffff801cf02c>] oom_kill_process+0xcc/0x248

[<ffffffff801cf70c>] __out_of_memory+0x134/0x1e8

[<ffffffff801cf834>] out_of_memory+0x74/0xe8

[<ffffffff801d37b4>] __alloc_pages_nodemask+0x5b4/0x5c8

[<ffffffff801e8278>] handle_mm_fault+0x3a8/0xfc8

[<ffffffff80169cd4>] do_page_fault+0x14c/0x3a8

[<ffffffff80100984>] ret_from_exception+0x0/0x10

 

Mem-Info:

DMA32 per-cpu:

CPU    0: hi:   42, btch:   7 usd:  38

CPU    1: hi:   42, btch:   7 usd:  40

CPU    2: hi:   42, btch:   7 usd:  31

CPU    3: hi:   42, btch:   7 usd:  39

Normal per-cpu:

CPU    0: hi:   18, btch:   3 usd:   2

CPU    1: hi:   18, btch:   3 usd:   2

CPU    2: hi:   18, btch:   3 usd:  15

CPU    3: hi:   18, btch:   3 usd:  16

active_anon:84311 inactive_anon:8022 isolated_anon:32

active_file:32 inactive_file:34 isolated_file:32

unevictable:1048 dirty:0 writeback:0 unstable:0

free:539 slab_reclaimable:286 slab_unreclaimable:1706

mapped:44 shmem:5826 pagetables:225 bounce:0

DMA32 free:1592kB min:1348kB low:1684kB high:2020kB active_anon:138248kB inactive_anon:2352kB active_file:152kB inactive_file:104kB unevictable:112kB isolated(anon):0kB isolated(file):0kB present:163708kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:1668kB slab_reclaimable:24kB slab_unreclaimable:4kB kernel_stack:0kB pagetables:268kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:480 all_unreclaimable? yes

lowmem_reserve[]: 0 69 69

Normal free:600kB min:584kB low:728kB high:876kB active_anon:198996kB inactive_anon:29736kB active_file:0kB inactive_file:32kB unevictable:4080kB isolated(anon):128kB isolated(file):128kB present:71280kB mlocked:0kB dirty:0kB writeback:0kB mapped:136kB shmem:21636kB slab_reclaimable:1120kB slab_unreclaimable:6820kB kernel_stack:816kB pagetables:632kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no

lowmem_reserve[]: 0 0 0

DMA32: 0*4kB 1*8kB 1*16kB 1*32kB 0*64kB 0*128kB 0*256kB 1*512kB 1*1024kB 0*2048kB 0*4096kB = 1592kB

Normal: 14*4kB 0*8kB 0*16kB 1*32kB 0*64kB 0*128kB 0*256kB 1*512kB 0*1024kB 0*2048kB 0*4096kB = 600kB

6981 total pagecache pages

0 pages in swap cache

Swap cache stats: add 0, delete 0, find 0/0

Free swap  = 0kB

Total swap = 0kB

127515 pages RAM

30210 pages reserved

56 pages shared

96519 pages non-shared

Out of memory: kill process 2975 (ls) score 5617 or a child

Killed process 2975 (ls)

Killed

 

 

Ian Miller
Senior Software Engineer II

tel +1 919 337 4183
fax +1 919 337 4149  

website | map | email

 



This email and attachments may contain privileged or confidential information intended only for the addressee(s) indicated. The sender does not waive any of its rights, privileges or protections respecting this information. If you are not the named addressee, an employee, or agent responsible for sending this message to the named addressee (or this message was received by mistake), you are not authorized to read, print, retain, copy or disseminate this message or any part of it. If received in error, please notify us immediately by e-mail, discard any paper copies and delete all electronic files of the email.

Computer viruses can be transmitted via email. The recipient should check this email and any attachments for viruses. Email transmission cannot be guaranteed to be secured or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender accepts no liability for any damage caused by any transmitted viruses or errors or omissions in the contents of this message.

Overture Networks, Inc. 637 Davis Drive, Morrisville, NC USA 27560 www.overturenetworks.com