Re: [Yaffs] data blocks lost between unmount and mount

Top Page
Attachments:
Message as email
+ (text/plain)
Delete this message
Reply to this message
Author: Blair Barnett
Date:  
To: Charles Manning
CC: yaffs
Subject: Re: [Yaffs] data blocks lost between unmount and mount
On Tue, 2005-10-04 at 13:14, Charles Manning wrote:
> On Wednesday 05 October 2005 04:51, Blair Barnett wrote:
>
> > The system has four partitions on the NAND (kernel image, root,
> > application, spare). After booting up and mounting root, application and
> > the spare partition, I create a file in the spare partition. Size
> > doesn't matter in this case. :-) A df command reveals that indeed some
> > blocks have been allocated to this file and all is well. Now if I remove
> > the file, a df command shows that indeed the blocks have been returned
> > to the free list (I know this is not exactly the case as the blocks have
> > all been "soft deleted" and waiting for GC). Now if I create another
> > file, the aforementioned blocks get GC'ed and df again shows that all is
> > right with the file system. If, however, I remove a file, then unmount
> > the file system, then mount it, df shows that the blocks the were "soft
> > deleted" are now marked as used (never to be returned to the free list,
> > as far as my tests show).
>
> Someone reported something similar recently, but did not seem to have come
> back with the requested debug info.
>
> I will do some investigation anyway.
>
> After you have deleted a file (even if it is only soft deleted), this is free
> space from a user perspective and should show up as being available under df.
> mount/unmount should not change df status.
>
> ie
> You should observe something like:
>
> * df. Shows 5MB free
> * dd to create a 1MB file
> *df. Shows 4MB free
> * rm file.
> * df shows 5MB free.

This works as stated above. Add the step of unmounting and mounting the
partition and you'll see that df shows 4 MB free (you lose the 1 MB of
soft deleted files *forever*, or at least until you do a
flash_eraseall).
>
> The only valid reason for df showing something else is if blocks got lost due
> to being retired because they had ECC errors.
>
> >
> > My question is shouldn't the unmount or mount code path cause GC to run
> > and return these "soft deleted" blocks back to the free list?
>
> Mounting/unmounting does not cause GC to run. GC only runs as a side effect of
> when you do any writing operations.
>
> Running gc at mount would only slow down mount time and would break the
> benefits of deferred GC.

Good point. This design decision is a very good one.
>
>
> >
> > Am I missing something? This unmount/mount block lost is filling our
> > application file system nightly.
>
> Do you get any warnings from mtd?

None at all.
>
> -- Charles
>

Hi Charles,

Here's a possible fix to the yaffs2 "losing blocks between
mounts/unmounts". This may not be the optimum way to fix it, but it
works for me. I just made yaffs_ScanBackwards() handle soft deleted
files just like yaffs_Scan(). Now my kernel works the same for both 512
and 2K page devices AND df works.

In looking through yaffs_Scan() and yaffs_ScanBackward(), it appears to
me as if you (or someone else) tried to get one scanning method to work,
which I think is admirable and "the right way to go"(TM). Since we call
yaffs_Scan() based on whether we're on a yaffs2 device or not, we can
probably safely remove all the "isYaffs2" tests in the yaffs_Scan()
code.

thanks again,

Blair



========================snip========================
diff -Naur yaffs2/yaffs_guts.c yaffs2_fixed/yaffs_guts.c
--- yaffs2/yaffs_guts.c 2005-09-19 22:08:50.000000000 -0700
+++ yaffs2_fixed/yaffs_guts.c   2005-10-04 15:56:56.000000000 -0700
@@ -1657,13 +1657,8 @@
                YBUG();
        }


-       /* TODO: Do we need this different handling for YAFFS2 and
YAFFS1?? */
-       if (obj->myDev->isYaffs2) {
-               unlinkOp = (newDir == obj->myDev->unlinkedDir);
-       } else {
-               unlinkOp = (newDir == obj->myDev->unlinkedDir
+       unlinkOp = (newDir == obj->myDev->unlinkedDir
                            && obj->variantType ==
YAFFS_OBJECT_TYPE_FILE);
-       }


        deleteOp = (newDir == obj->myDev->deletedDir);


@@ -4194,16 +4189,12 @@


yaffs_AddObjectToDirectory(parent, in);

-                                       if (0 && (parent ==
dev->deletedDir ||
-                                                 parent ==
dev->unlinkedDir)) {-                                              
in->deleted = 1;        /* If it is unlinked at start up then it wants
deleting */
-                                               dev->nDeletedFiles++;
-                                       }
                                        /* Note re hardlinks.
-                                        * Since we might scan a
hardlink before its equivalent object is scanned
-                                        * we put them all in a list.
-                                        * After scanning is complete,
we should have all the objects, so we run through this
-                                        * list and fix up all the
chains.                                                                                 
+                                        * Since we might scan a
hardlink before its equivalent
+                                        * object is scanned we put them
all in a list.
+                                        * After scanning is complete,
we should have all
+                                        * the objects, so we run
through this list and fix up
+                                        * all the chains.
                                         */


                                        switch (in->variantType) {
@@ -4762,22 +4753,17 @@



yaffs_AddObjectToDirectory(parent, in);

-                                       if ((parent == dev->deletedDir
||
-                                            parent ==
dev->unlinkedDir)) {
-                                            /* If it is unlinked at
start up then it wants deleting */
-                                               in->deleted = 1;
-                                       }
-
                                        if (oh->isShrink) {
                                                /* Mark the block as
having a shrinkHeader */
                                                bi->hasShrinkHeader = 1;
                                        }


                                        /* Note re hardlinks.
-                                        * Since we might scan a
hardlink before its equivalent object is scanned
-                                        * we put them all in a list.
-                                        * After scanning is complete,
we should have all the objects, so we run
-                                        * through this list and fix up
all the chains.
+                                        * Since we might scan a
hardlink before its equivalent
+                                        * object is scanned we put them
all in a list.
+                                        * After scanning is complete,
we should have all
+                                        * the objects, so we run
through this list and
+                                        * fix up all the chains.
                                         */


                                        switch (in->variantType) {