Andrew, Yang,
On Wednesday 05 November 2008, Andrew McKay wrote:
> Yang Chen wrote:
> > Andrew,
> >
> > Thanks for your info. I will try the fix. Btw , have you
> > tested below fix with more cases?
>
> I've been running the Kernel on a few of our systems with this
> fix and haven't found any weird issues yet. It was more
> important that ls wasn't getting stuck in a loop. Our system
> would lock up,.
>
> > I searched archived mails and found an old mailloop
> > regarding the rewind change . Looks like that was for rm
> > -rf issues of busybox ...
>
> If rm -rf did leave something behind, that's less of a problem
> for us. At least we could modify our scripts to handle that
> case. Check for the existence of the directory after it's
> removal and try again.
>
> My coworker had done some digging for info for me and came up
> with this:
>
> --------------------------------------------------------------
>----------------- Some stuff from the posix standard:
>
> "If a file is removed from or added to the directory after the
> most recent call to opendir() or rewinddir(), whether a
> subsequent call to readdir() returns an entry for that file is
> unspecified."
>
> In other words, "readdir" is not expected to care if new files
> are added to a directory while it is running.
>
> <http://www.opengroup.org/onlinepubs/009695399/functions/readd
>ir.html>
>
> Rm is not guaranteed to succeed in the posix std.
>
> "If this fails for any reason, rm shall write a diagnostic
> message to standard error, do nothing more with the current
> file, and go on to any remaining files."
>
> <http://www.opengroup.org/onlinepubs/009695399/utilities/rm.ht
>ml>
>
> Also,
>
> "Some implementations do not permit the removal of the last
> link to an executable binary file that is being executed; see
> the [EBUSY] error in the unlink() function defined in the
> System Interfaces volume of IEEE Std 1003.1-2001. Thus, the rm
> utility can fail to remove such files."
> --------------------------------------------------------------
>-----------------
>
> So rm -rf leaving stuff behind doesn't break posix standards
> and is definitely less of an issue than readdir never
> returning NULL and putting applications into endless loops, or
> causing them to use up all the memory in the system (ls
> buffers everything before trying to write it out to the
> terminal).
>
> Andrew
The problem IIRC is that Yaffs does not have a real directory
structure per se. Nearly all other filesystems have some form
of on-media directory structure. When reading a directory
(opendir/readdir) one normally moves over the entries in a
directory just like reading a file. When an entry is removed
from a directory, the slot occupied by the entry is left in
place, marked as free. The readdir operation simply skips over
the free slots.
This is not how Yaffs works, Yaffs directories are more like
in-core linked lists. When a directory entry is removed, it is
removed from the list and all subsequent entries appear to move
up the list.
When reading a directory, there is a notion of a current
position, a cursor -- this paradim does not map easily onto a
linked list. It works okay when the directory has a real
structure that still contains the slot that was occupied by a
removed entry.
The earlier change to Yaffs, rewinding the 'cursor', was to help
fix a problem with 'rm -rf' and the like. If the 'rm' command
only performs a single pass over a directory, think linked-list,
then it may miss an entry if its 'cursor' is to the right of an
removed entry. As the 'rm' command 'unlink's entries, the list
shrinks and entries move to the left, but the 'cursor' keeps
advancing. The solution to 'rm -rf' problem is to repeatedly
pass over the directory until it stops succeeding at 'unlink'ing
entries. Apparently, busybox 'rm' does (did?) not do this.
Busybox is very often used with Yaffs on Linux, and but Yaffs is
perceived to be the odd one (filesystem) out, thus the push for
the earlier 'fix'. But Yaffs is not alone, there are several
filesystems that have similar issues -- usually caused by the
same underlying issue -- that directories may dynamic objects,
not simply flat files.
Careful thought should be put into this issue and any changes.
Tweaking for one case may easily break another. Seams like both
Yaffs AND busybox could be improved here.
-imcd
(text/plain)