System Design: BigTable

1.4S analysis

scenario: read / write
service: client - BigTable - GFS
store: NoSQL
scale:

2.A workable solution

1.Read:

2.Write:

3.A better and detailed solution

1.Read:

[Explain for details]

1. Distributed lock server
1. provide lock service
   (Chubby in Google: http://bit.ly/1Pukiyt
   Zookeeper in Apache: http://bit.ly/1TOWIsR)
2. manage metadata (the consistent hashmap)
2. Master, lock server and user
1. Master: in this framework, master does not maintain the consistent hashmap anymore, it 1) manage the servers health and 2) shard the files
2. Lock Server: 1) maintain the metadata 2)lock the key
3. Communication:
   Communication (user<->lock server): lock, slave server id
   Communication (lock server<->master): when a slave server fail, the master needs to assign a new one, copy the data and update the consistent table in the lock server.
   No communication (user<->master)
3. Tablet
4. SSTable
5. Optimize read from GFS
1. index
2. bloom filter

2.Write:

1. The writing process in tablet server:
1. write-ahead log: http://www.larsgeorge.com/2010/01/hbase-architecture-101-write-ahead-log.html
2. write memory
3. asynchronously write the SSTable in memory to GFS
1. sort
2. index
3. write bloom filter