On 02/02/2012 05:33 PM, Charles Manning wrote:
> On Tuesday 17 January 2012 12:39:43 Charles Manning wrote:
>> Hello Peter
>>
>> On Tuesday 17 January 2012 08:13:21 Peter Barada wrote:
>>> I need to modify YAFFS to trigger garbage collection on a block that has
>>> too many page reads between erases.
>>>
>>> This is due to the characterisation of the Micron NAND
>>> (MTMT29C4G48MAPLCJI6) I'm using where it can only have ~20K reads
>>> between erasures to maintain UBER below 10E-14. The NAND itself has an
>>> internal 4-bit ECC engine, but I can't extract any useful information
>>> from the ECC due to the lack of bit error counts (i.e. the chip can only
>>> tell me that a read required some level of correction but not the amount
>>> of correction, and worse the rate of corrections indicated by the chip
>>> is too high to employ effective strike counting).
>> Oh how these silicon vendors seem to take glee in causing us pain!
>>
>>> So I need to track the page reads in a block and if they reach a limit
>>> then cause the block to be garbage collected (i.e. copy out all the
>>> useful data and erase the block).
>> You just need to run the block through the garbage collector.
>>
>> But first the block needs to be in FULL state. Now that is most likely the
>> case, but it is possible to contrive some cases where this might not be
>> (eg. write a couple of pages then read just those pages a million times).
>>
>> That can be achieved by some code in the spirit of
>>
>> if(block state == ALLOCATING)
>> yaffs_skip_rest_of_block(dev);
>>
>>
>> 2. Then I think it will be ok to just call yaffs_gc_block() while locked.
>>
>> yaffs_gc_block(dev, block_i_want_to_refresh, 1);
>>
>>
>> -or- cleverer
>>
>> Or you could just set dev->gc_block = block_i_want_to_refresh and the
>> garbage collector will pick it up and sort it out in the next gc.
>>
>> Of course that has the potential problem of two blocks needing refreshing
>> and one interrupting the other.
>>
>> Perhaps the two ideas can be melded as follows:
>>
>> if(block state == ALLOCATING)
>> yaffs_skip_rest_of_block(dev);
>>
>> if(dev->gc_block)
>> yaffs_gc_block(dev, block_i_want_to_refresh, 1);
>> else
>> dev->gc_block = block_i_want_to_refesh;
>>
>> - or - cleverist
>>
>> Add a list of blocks needing refresh to dev and in yaffs_check_gc() add a
>> new clause for getting a new gc block
>>
>> if (dev->gc_block< 1&& refresh list not empty) {
>> dev->gc_block = grab first in list;
>> dev->gc_chunk = 0;
>> dev->n_clean_ups = 0;
>> }
>>
>>> Any help is appreciated!
>> Hope that helps....
>>
>> Charles
>>
> Hi Peter
>
> I was wondering how you got on with the block refreshing.
>
> Last night I was thinking about something unrelated and suddenly realised that
> I think there might be a corner case where data written in a certain pattern
> could come back from the dead if you just do what I suggest above.
>
> Consider the following sequence:
> * create file
> * write 3MB
> * truncate to 1MB
> * seek to 3MB
> * write 1MB of data
>
> That should now give you a file that is 4MB in size, but only has valid data
> in the first and last MB. The 2MB in the middle is a "hole" that should be
> read as zeros.
>
> YAFFS2 handles this by creating a shrink header to mark the beginning of the
> hole.
>
> If you run the garbage collector on the block containing the shrink header
> (thus deleting the shrink header) and then do an unclean shut down, the
> scanner will no longer know that there is a hole there and any remaining
> chunks in that hole will come back to life.
>
> This should only be an issue if you do weird things as shown above.
>
> yaffs mitigates against this by only garbage collecting a block with shrink
> headers if they meet certain criteria. See yaffs_block_ok_for_gc().
>
> Thus I think the best approach for doing the refreshing is to introduce a new
> mechanism for the handling of prioritised blocks that allows multiple blocks
> to be held for refreshing. If you need a block refreshed then just add it to
> the list and the gc mechanism will handle it when appropriate.
>
> Does that sound valuable?
>
> Charles
Yes it does. I've hacked away on it within my linux-3.0 kernel:
0) Add "struct list_head refresh_entry;" to struct yaffs_block_info,
"struct list_head refresh_head;" to struct yaffs_device to have a list
of potential blocks to refresh.
1) when yafs_handle_chunk_error() add the following - if the block
requires refreshing:
/* This chunk is in a stale block (MTD returned -ESTALE)
* First stop allocating in the block and add it to
* the refresh ring */
if (bi->block_state == YAFFS_BLOCK_STATE_ALLOCATING)
yaffs_skip_rest_of_block(dev);
yaffs2_refresh_list_add(dev, bi);
2) in yaffs_init_blocks(), add:
for (i=0; i<n_blocks; ++i)
INIT_LIST_HEAD(&dev->block_info[i].refresh_entry);
3) In yaffs_initialise_device() add:
INIT_LIST_HEAD(&dev->refresh_head);
4) In yaffs_check_gc(), add the following before the comment "If we
don't already have a block being gc'd then see if we should start another":
/* If we don't already have a block being gc'd then see if
* there's one that's marked for refresh by -ESTALE */
if (dev->gc_block < 1 && yaffs2_refresh_list_not_empty(dev)) {
dev->gc_block = yaffs2_refresh_list_next(dev);
if (dev->gc_block > 0) {
dev->gc_chunk = 0;
dev->n_clean_ups = 0;
}
}
5) In the do/while loop the above code is in, add
"yaffs2_refresh_list_not_empty(dev) ||" to the exit condition so it will
loop through garbage collecting all the blocks in the refesh ring.
Of course I'm omitting the code that deals with putting a block on the
refresh list, remove one, and show if its empty, but this should give
you an idea of what its doing.
So far this looks to work, when the trigger is set 20K reads, then it
looks to do another ~1500 reads before the block gets garbage collected
(and erased). I'm wondering how to get the GC to trigger quicker (i.e.
when the yaffs_gross_unlock() is called) - have the garbage collector
immediately run if there's a block to refresh.
I'll have to go back and modify the code to deal with blocks that can't
be currently refreshed (as in the weird caseyou outlined above), and
then have code tha tlooks in the refresh list for the *first* block that
can be refreshed instead of using "is the refresh list not empty, go
refresh the first block in the list". I'll try to put together a set of
patches against the current yaffs GIT tree and send them to the list -
hopefully I'll have something together in a few days...
Another thing that I'm thinking about is how to modify yaffs to do
static wear levelling - to maximize the life of the part erases should
be evenly spread across all the blocks in the partition, so any static
data that isn't read (while other data is updated) can result in blocks
that are hardly erased while others are erased multiple times. Any
ideas how to have it look at the number of erases done on each block and
if the high/low counts get too far apart then "refresh" the low erased
blocks? This will increase the total number of erases done, but I think
it should do better wear levelling than it currently does. What would
be really nice is to have block summaries on the NAND (including
read/erase counts) that live across an improper shutdown (when the
checkpoint is lost) to track read counts over multiple power cycles
instead of starting at zero as i currently do. I haven't figured out
how to do this - could it be incorporated in the block summary changes
you mentioned a bit ago?
Thanks!
--
Peter Barada
pete.barada@logicpd.com
(text/plain)