[tahoe-dev] [tahoe-lafs] #738: failure in block hash tree

[tahoe-lafs]

#738: failure in block hash tree
-------------------------------+--------------------------------------------
     Reporter:  midnightmagic  |        Type:  defect       
       Status:  new            |    Priority:  critical     
    Milestone:  1.5.0          |   Component:  code-encoding
      Version:  1.4.1          |    Keywords:  integrity    
Launchpad_bug:                 |  
-------------------------------+--------------------------------------------

Comment(by warner):

 I've looked at that report and compared it against the scrubber that I
 wrote (a server-side share verification tool). It looks like your client
 is generating a different hash for the first data block than it's supposed
 to. The incident report contains a copy of the first 50 bytes and the last
 50 bytes of the block, and they match what I'm getting out of the share.

 So, either your client is incorrectly computing the SHA256d hash of that
 43kB-ish data, or it is using a block of data that is corrupted somewhere
 in the middle. Your client seems to compute the rest of the hash tree
 correctly (and I think you might have said that pycryptopp tests pass on
 this platform), so it seems like SHA256d is working in general. So that
 points to either the wrong hash tag (in allmydata.util.hashutil), or some
 sort of transport-level error that is corrupting or inserting/deleting
 data in the middle of the block.

 I've just pushed some known-answer-tests to confirm that
 allmydata.util.hashutil is working correctly: could you pull a trunk tree,
 build, and then specifically run "make test TEST=allmydata.test.test_util"
 ? I know that an earlier test is hanging on your system; by running the
 later test_util directly, we can rule out this one hypothesis.

 If that passes, the next step will be to patch the download code to save
 the full block to disk, so we can examine it and see if it matches what
 it's supposed to be.

-- 
Ticket URL: <http://allmydata.org/trac/tahoe/ticket/738#comment:4>
tahoe-lafs <http://allmydata.org>
secure decentralized file storage grid