Hi Charles,

Em 02/06/14 00:58, Charles Manning escreveu:

On Sun, Jun 1, 2014 at 9:06 AM, Leonardo Rodrigues <leolistas@solutti.com.br> wrote:

    Hello Everyone,

    I'm facing a critical problem with yaffs2 partition when using OpenWRT on RouterBoards, specially RB450 and RB450G, which are the ones i have to test/use.

    After normal operations of writing and deleting files, space seems to completly got lost on the partition until the point i get 'no space left on the device' and have to reflash the board to get it operational again. On RB450 board the space seems to get lost during reboots, while on RB450Gs it get lost on reboots as well as normal write/delete operations as well.

From the description you give here, it is moat likely a problem with bad block detection.

    As i have used a LOT of RouterBoards for delivering several different solutions, it's known to me that several of those NAND chips cames with badblocks marked from factory and this is NOT a problem. However, this specific RouterBoard 450G i'm using for the tests seems to not have any single bad block marked. During the boot, no badblock is shown. Given my experience with routerboards, badblocks usually are shown here. So, maybe the situation i'm facing is not badblock related.

[    1.380000] NAND flash driver for RouterBoard 4xx series version 0.2.0
[    1.380000] NAND device: Manufacturer ID: 0xad, Chip ID: 0x76 (Hynix NAND 64
MiB 3,3V 8-bit), 64MiB, page size: 512, OOB size: 16
[    1.400000] Scanning device for bad blocks
[    2.290000] Creating 3 MTD partitions on "NAND 64MiB 3,3V 8-bit":
[    2.290000] 0x000000000000-0x000000040000 : "booter"
[    2.300000] 0x000000040000-0x000000400000 : "kernel"
[    2.310000] 0x000000400000-0x000004000000 : "rootfs"
[    2.310000] mtd: partition "rootfs" set to be root filesystem
[    2.320000] split_squashfs: no squashfs found in "NAND 64MiB 3,3V 8-bit"

    I have found other users claiming similar problems with RouterBoards models as well, despite i can only test on RB450 and RB450G.

    As i'm facing the problem on OpenWRT, i have already opened a ticket to its bugtraq system, which had not received any attention yet.

    With latest OpenWRT svn versions, i think things will be hard to debug on yaffs part, as it seems that's for compiling 3.10 kernel, the /proc/yaffs files were disabled (done on OpenWRT revision 37285)

https://dev.openwrt.org/changeset/37285

    I have also download the revision 37000, which still builds on kernel 3.9 and have the /proc/yaffs files, and the problem occurs exactly the same way as related on my bugtraq ticket to the OpenWRT team. If needed, i can provide /proc/yaffs contents between the tests i have made, just let me know in which points i can provide that information.

If you provide two or three /proc/yaffs dumps that would be helpful... one near the start, one a few tests in and one towards the end.

    I have changed my test loop script to dump /proc/yaffs and /proc/yaffs_stats after each run and let it run for a lot of times. Tests, as the previous one, were done on a fresh installed OpenWRT install, that means, no previous 'problems' on the yaffs partition nor 'lost space'.

    Please remember that i'm NOT running latest OpenWRT svn versions, as those uses kernel 3.10 which had the /proc yaffs files disabled. I'm running the r37000 which already runs on kernel 3.9 and, so, still have the /proc files. Anyway, even on the latest svn ones, the problem is absolutely the same as on r37000.

    After run #1, i had:

run # 1
Filesystem           1K-blocks      Used Available Use% Mounted on
rootfs                   61440      5692     55748   9% /

dumping /proc/yaffs
Multi-version YAFFS built:May 30 2014 16:40:29

Device 0 "rootfs"
start_block.......... 0
end_block............ 3839
total_bytes_per_chunk 512
use_nand_ecc......... 1
no_tags_ecc.......... 0
is_yaffs2............ 0
inband_tags.......... 0
empty_lost_n_found... 0
disable_lazy_load.... 0
refresh_period....... 500
n_caches............. 10
n_reserved_blocks.... 5
always_check_erased.. 0

data_bytes_per_chunk. 512
chunk_grp_bits....... 0
chunk_grp_size....... 1
n_erased_blocks...... 3108
blocks_in_checkpt.... 0

n_tnodes............. 1108
n_obj................ 651
n_free_chunks........ 111678

n_page_writes........ 10406
n_page_reads......... 9049
n_erasures........... 0
n_gc_copies.......... 0
all_gcs.............. 0
passive_gc_count..... 0
oldest_dirty_gc_count 0
n_gc_blocks.......... 0
bg_gcs............... 0
n_retired_writes..... 0
nRetireBlocks........ 0
n_ecc_fixed.......... 0
n_ecc_unfixed........ 0
n_tags_ecc_fixed..... 0
n_tags_ecc_unfixed... 0
cache_hits........... 0
n_deleted_files...... 11
n_unlinked_files..... 15
refresh_count........ 0
n_bg_deletions....... 0

dumping /proc/yaffs_stats
0, 111678, 99456, 0, 0, 651, 1108

    after run #50:

run # 50
Filesystem           1K-blocks      Used Available Use% Mounted on
rootfs                   61440      5692     55748   9% /

dumping /proc/yaffs
Multi-version YAFFS built:May 30 2014 16:40:29

Device 0 "rootfs"
start_block.......... 0
end_block............ 3839
total_bytes_per_chunk 512
use_nand_ecc......... 1
no_tags_ecc.......... 0
is_yaffs2............ 0
inband_tags.......... 0
empty_lost_n_found... 0
disable_lazy_load.... 0
refresh_period....... 500
n_caches............. 10
n_reserved_blocks.... 5
always_check_erased.. 0

data_bytes_per_chunk. 512
chunk_grp_bits....... 0
chunk_grp_size....... 1
n_erased_blocks...... 793
blocks_in_checkpt.... 0

n_tnodes............. 1111
n_obj................ 654
n_free_chunks........ 111675

n_page_writes........ 513144
n_page_reads......... 444342
n_erasures........... 13381
n_gc_copies.......... 43
all_gcs.............. 93617
passive_gc_count..... 93617
oldest_dirty_gc_count 0
n_gc_blocks.......... 13381
bg_gcs............... 0
n_retired_writes..... 0
nRetireBlocks........ 0
n_ecc_fixed.......... 0
n_ecc_unfixed........ 0
n_tags_ecc_fixed..... 0
n_tags_ecc_unfixed... 0
cache_hits........... 0
n_deleted_files...... 14
n_unlinked_files..... 64
refresh_count........ 0
n_bg_deletions....... 0

dumping /proc/yaffs_stats
0, 111675, 25376, 0, 0, 654, 1111

    after run #100:

run # 100
Filesystem           1K-blocks      Used Available Use% Mounted on
rootfs                   61440      5692     55748   9% /

run # 100
dumping /proc/yaffs
Multi-version YAFFS built:May 30 2014 16:40:29

Device 0 "rootfs"
start_block.......... 0
end_block............ 3839
total_bytes_per_chunk 512
use_nand_ecc......... 1
no_tags_ecc.......... 0
is_yaffs2............ 0
inband_tags.......... 0
empty_lost_n_found... 0
disable_lazy_load.... 0
refresh_period....... 500
n_caches............. 10
n_reserved_blocks.... 5
always_check_erased.. 0

data_bytes_per_chunk. 512
chunk_grp_bits....... 0
chunk_grp_size....... 1
n_erased_blocks...... 792
blocks_in_checkpt.... 0

n_tnodes............. 1111
n_obj................ 654
n_free_chunks........ 111675

n_page_writes........ 1026196
n_page_reads......... 956886
n_erasures........... 29398
n_gc_copies.......... 95
all_gcs.............. 205683
passive_gc_count..... 205683
oldest_dirty_gc_count 0
n_gc_blocks.......... 29398
bg_gcs............... 0
n_retired_writes..... 0
nRetireBlocks........ 0
n_ecc_fixed.......... 0
n_ecc_unfixed........ 0
n_tags_ecc_fixed..... 0
n_tags_ecc_unfixed... 0
cache_hits........... 0
n_deleted_files...... 14
n_unlinked_files..... 114
refresh_count........ 0
n_bg_deletions....... 0

dumping /proc/yaffs_stats
0, 111675, 25344, 0, 0, 654, 1111

    after run #150 (and last):

run # 150
Filesystem           1K-blocks      Used Available Use% Mounted on
rootfs                   61440      5692     55748   9% /

dumping /proc/yaffs
Multi-version YAFFS built:May 30 2014 16:40:29

Device 0 "rootfs"
start_block.......... 0
end_block............ 3839
total_bytes_per_chunk 512
use_nand_ecc......... 1
no_tags_ecc.......... 0
is_yaffs2............ 0
inband_tags.......... 0
empty_lost_n_found... 0
disable_lazy_load.... 0
refresh_period....... 500
n_caches............. 10
n_reserved_blocks.... 5
always_check_erased.. 0

data_bytes_per_chunk. 512
chunk_grp_bits....... 0
chunk_grp_size....... 1
n_erased_blocks...... 793
blocks_in_checkpt.... 0

n_tnodes............. 1111
n_obj................ 654
n_free_chunks........ 111675

n_page_writes........ 1539248
n_page_reads......... 1469462
n_erasures........... 45416
n_gc_copies.......... 147
all_gcs.............. 317756
passive_gc_count..... 317756
oldest_dirty_gc_count 0
n_gc_blocks.......... 45416
bg_gcs............... 0
n_retired_writes..... 0
nRetireBlocks........ 0
n_ecc_fixed.......... 0
n_ecc_unfixed........ 0
n_tags_ecc_fixed..... 0
n_tags_ecc_unfixed... 0
cache_hits........... 0
n_deleted_files...... 14
n_unlinked_files..... 164
refresh_count........ 0
n_bg_deletions....... 0

dumping /proc/yaffs_stats
0, 111675, 25376, 0, 0, 654, 1111

    and rebooting immediately after run #150 (used space went from 5.7Mb to almost 48Mb)

root@OpenWrt:/# df
Filesystem           1K-blocks      Used Available Use% Mounted on
rootfs                   61440     47844     13596 78% /

root@OpenWrt:/# cat /proc/yaffs
Multi-version YAFFS built:May 30 2014 16:40:29

Device 0 "rootfs"
start_block.......... 0
end_block............ 3839
total_bytes_per_chunk 512
use_nand_ecc......... 1
no_tags_ecc.......... 0
is_yaffs2............ 0
inband_tags.......... 0
empty_lost_n_found... 0
disable_lazy_load.... 0
refresh_period....... 500
n_caches............. 10
n_reserved_blocks.... 5
always_check_erased.. 0

data_bytes_per_chunk. 512
chunk_grp_bits....... 0
chunk_grp_size....... 1
n_erased_blocks...... 793
blocks_in_checkpt.... 0

n_tnodes............. 7168
n_obj................ 654
n_free_chunks........ 27390

n_page_writes........ 0
n_page_reads......... 8244
n_erasures........... 0
n_gc_copies.......... 0
all_gcs.............. 0
passive_gc_count..... 0
oldest_dirty_gc_count 0
n_gc_blocks.......... 0
bg_gcs............... 0
n_retired_writes..... 0
nRetireBlocks........ 0
n_ecc_fixed.......... 0
n_ecc_unfixed........ 0
n_tags_ecc_fixed..... 0
n_tags_ecc_unfixed... 0
cache_hits........... 0
n_deleted_files...... 0
n_unlinked_files..... 14
refresh_count........ 0
n_bg_deletions....... 0

root@OpenWrt:/# cat /proc/yaffs_stats
0, 27390, 25376, 0, 0, 654, 7168

You might also consider using inband tags to see if this makes the problem go away.

i dont have a clue how to add this mount option on the OpenWRT init files. I'll look for it.

-- 


	Atenciosamente / Sincerily,
	Leonardo Rodrigues
	Solutti Tecnologia
	http://www.solutti.com.br

	Minha armadilha de SPAM, NÃO mandem email
	gertrudes@solutti.com.br
	My SPAMTRAP, do not email it