distributed system summary Jan 2

distributes system is a capstone course of NYU CS master program, which I pick in the first semester.   

the course web site is  http://www.news.cs.nyu.edu/~jinyang/fa10/

this course includes several papers for reading and 8 labs, among which the labs are quite challenging.

the overview shows as follows.

---------------------------------------------------------------------------

In this sequence of labs, you'll build a multi-server file system called Yet-Another File System (yfs) in the spirit of Frangipani. At the end of all the labs, your file server architecture will look like this:

 

You'll write a file server process, labeled yfs above, using the FUSE toolkit. Each client host will run a copy of yfs. yfs will appear to local applications on the same machine by registering via FUSE to receive file system events from the operating system. The yfs extent server will store all the file system data on an extent server on the network, instead of on a local disk. yfs servers on multiple client hosts can share the file system by sharing a single extent server.

------------------------------------------------------------------------------------------------------------

lets start with lab1: 

LAB 1 :

familiarize the RPC calls,  pthread mutex.

key points:

1, use RPC calls.   in the client side, use ".call()" and in the server side implement RPC handler. in this lab, the server side need not to run to completion. 

2, familiarize the pthread mutex. the lock_server achieve a simple lock and unlock by using pthread. one caveat is that before we return and exit one function, we should release all mutex we are holding!!  this is a common mistake.   

 

we should also modify rpc.cc to achieve at-most-once delivery. 

here we don't talk about the specific implementation. instead, we point out several general regulation in coding. 

1, template. 

in the rpc.h, rpcc class

代码

class rpcc : public changer {

// ..........
struct TO{...};

template < class R >
int call_m(unsigned int proc, marshall & req, R & r, TO to)

template < class R >
int call(unsigned int proc, R & r, TO to = tp_max)

// other template

}

template < class R > int
rpcc::call_m(unsigned int proc, marshall & req, R & r, TO to)
{
unmarshall u;
int intret = call1(proc, req, u, to);
// ......
}

// here we use marshall and unmarshall
template < class R > int
rpcc::call(unsigned int proc, R & r, TO to)
{
marshall m;
return call_m(proc, m, r, to);
}

template < class R, class A1 > int
rpcc::call(unsigned int proc, const A1 & a1, R & r, TO to)
{
marshall m;
m << a1;
return call_m(proc, m, r, to);
}
// ......

 

2, mashall and unmarshall 

is used to envelop the data. 

代码

class marshall{
// ........
// in mashall, we can transfer marshall to string
std:: string str() {
return get_content();
}
// .....
}

marshall & operator << (marshall & , unsigned int );
marshall & operator << (marshall & , int );
// mashall vector
template < class C > marshall &
operator << (marshall & m, std::vector < C > v)
{
m << (unsigned int ) v.size();
for (unsigned i = 0 ; i < v.size(); i ++ )
m << v[i];
return m;
}

 

and unmarshall has the same principle  

template < class C > unmarshall &
operator >> (unmarshall & u, std::vector < C > & v)
{
unsigned n;
u >> n;
for (unsigned i = 0 ; i < n; i ++ ){
C z;
u >> z;
v.push_back(z);
}
return u;
}

 

in the latter lab, we also use marshall and unmarhsall to transfer data 

代码

int extent_server::getattr(extent_protocol::extentid_t id, extent_protocol::attr & a)
{
// You replace this with a real implementation. We send a phony response
// for now because it's difficult to get FUSE to do anything (including
// unmount) if getattr fails.
// .......this is a good example for how to use marshall and unmarshall
pthread_mutex_lock( & extent_mutex);
std:: string temp = meta_data[id];
pthread_mutex_unlock( & extent_mutex);

unmarshall at(temp);
extent_protocol::attr attr_temp;
at >> attr_temp; // unmarshall, convert string to attr

a.size = attr_temp.size;
a.atime = attr_temp.atime;
a.mtime = attr_temp.mtime;
a.ctime = attr_temp.ctime;


return extent_protocol::OK;
}

 

 

3, pay attention to the scope of the lock. 

i.e,  add_reply and check_duplicate_update may be not consistent. 

scopedlock:

代码

ifndef Foundation_ScopedLock_INCLUDED
#define Foundation_ScopedLock_INCLUDED


#ifndef Foundation_Foundation_INCLUDED
#include " Foundation/Foundation.h "
#endif

Foundation_BEGIN

template < class M >
class ScopedLock
/// A class that simplifies thread synchronization
/// with a mutex.
/// The constructor accepts a Mutex and locks it.
/// The destructor unlocks the mutex.
{
public :
inline ScopedLock(M & mutex): _mutex(mutex)
{
_mutex. lock ();
}
inline ~ ScopedLock()
{
_mutex.unlock();
}

private :
M & _mutex;

ScopedLock();
ScopedLock( const ScopedLock & );
ScopedLock & operator = ( const ScopedLock & );
};

Foundation_END

#endif // Foundation_ScopedLock_INCLUDED

 

 

转载于:https://www.cnblogs.com/ggppwx/archive/2011/01/03/1924476.html