The reason of designing GFS is due to the fact that nowadays data storage have some new features.
1. Component failure are normal.
2. The data files are huge.
3. Most files are used for appending more data rather than rewrite existed data.
4. Concurrently write the same file.
The four main changes are significant and need to be solved.
2) What is the solution’s main idea (nugget)?
It contains mainly 3 roles.
1. The GFS Master mainly stores all file system metadata (single).
2. The GFS Chunkserver stores the chunks(Multiple)
3. The GFS client is used as agent for requesting data on the Server.
3) Why is solution different from previous work?
The fils system has been a very old problem.
The workload is different from previous system. The main workload difference are as follows: more component failures, frequently read & append huge files, etc. These workload characteristics lead to the GFS assumptions to be different from other systems.
Based on the different workload. GFS mainly differs from previous works in following parts. These workload characteristics lead to the GFS assumptions to be different from other systems.
1. GFS do not have to cache below the file system interface.
2. GFS still keeps centralized control pattern rather than some distributed file system.
3. GFS focus on Fault tolerance rather than POSIX compliant system.
4. GFS chunk servers use lazily allocated fixed-size chunks rather than variable-length objects (e.g. NASD).
4) Does the paper (or do you) identify any fundamental/hard trade-offs?
In order to keep the system's robust to component failures, every chunk of data has replicas (in total 3). The replicas increases the system overhead on data consistency. In addition, I think it is not efficient, since every piece of data should be write three times and take 3 times of its data size on storage.
Definitely. Hadoop is using HDFS which is based on GFS system design. And Hadoop is widely-deploied in real-world big data analysis systems.
No comments:
Post a Comment