Using NetBSD based sockets, implement a TCP client and TCP Server to demonstrate trivially secure file download from server.

Environment/Tools required: Linux RHEL4/5/6 and gcc/g++ compiler and vi editor (recommended) OR Windows XP/Vista/7/8 with Visual studio.

Learning Objective for student:

Understand Protocol messaging and parsing. Manage multiple connections at a server side Manage the state of a connection at both client and server side. Ability to use NetBSD Sockets via C API. Binary vs Text and practical issues with Endianness ! Ability to effectively use fread, fwrite APIs for file management can also use read/write system calls. Understanding Standards and compatibility (and the inherent difficulty!) between different (not differing!) implementations. Note: This is a group assignment. Each group can have maximum of 10 students. If group of 10 students is opted, 5 students may contribute to the client and another 5 students may contribute to the server. but put together at a minimum, the expected result is a group's server should work with its own client. If a group's server/client are compatible with other group's client/servers, it will receive more credit. ** Each group is expected to write both the client and a server." -1- Details: The messaging mechanism is based on Type-Length-Value (TLV) The Message Format is TYPE Field ---- Length Field --- Value field --------------- ---- 8 bytes --- 4 Bytes int ---- buffer of length bytes ---- Note: If Length Field is ZERO, then the Value field is empty which is still a perfectly valid message. eg. LOGINOK message - See details below. All types are of size 8 chars and always uppercase. Valid Types for Client messages are: LOGIN, AUTH, GETFILE Valid Types for Server messages are: NEEDAUTH, LOGINOK, DATA, ERROR As Type field is 8 bytes, it will employ 'SPACE' as padding. i.e. LOGIN will be sent as "LOGIN " The message exchange details are as below: Client connects to the server. LOGIN is the first message sent by the client. If server requires authentication, it replies with NEEDAUTH message. If server does not require authentication, it replies with LOGINOK message. Client sends back AUTH message as a reply to NEEDAUTH message Server replies with LOGINOK if authentication successful, else replies will reply with ERROR and close the connection. Client if login was ok, i.e. received LOGINOK message, can send at any time, a file download request by sending the message GETFILE. Note: Only outgoing request must be in the network pipe. Upon receiving a GETFILE message, if server finds a matching file it can send the file contents back in the DATA message. If no matching file, it may send back an ERROR message. Note, any time the client can close the connection and go away, server needs to detect this and close the connection on its side as well. eg. if while server is sending a file, the client goes away, server needs to stop then and there. -2- The Client upon receiving the DATA message should store the contents appropriately into a file for later viewing. Server never advertises the file list on the server (unlike SCP/SFTP/FTP !) so clients need to know the file names a simple way is to query the user Note: A fancy UI is not required, rather a basic working UI with scanf() or fgets() should suffice. Note about GETFILE: The GETFILE message can send as value a full path name (absolute) or a relative path name. with "/" as directory separator. eg. If "hello/details.txt" is sent, server is supposed to look in its working directory the given path of the file. So in this case "hello/details.txt" and "details.txt" may refer to completely different files. Command Line arguments: ./TSServer ./TSClient Note about Auth Mechanism: Any 2-way auth mechanism can be employed, trivially a Salted Password hash based mechanism can be employed. This avoids replay attacks! NOTE: * If the environment chosen is Linux, it comes with a tool called 'md5sum' which can be used to generate the Hashes on both client side and server side. * The server can store the list of passwords in some file. (Bad! Not secure, but ok for this barebones version) * Both the server and client programs may run in the same machine during development, but for testing it would be ideal to run them in different machines, to see how a client/server reacts when network goes away. ( Intentionally pull out the ethernet cable when the client is downloading a big file!) -3-

Message Details

There are totally 7 messages - Syntax/Structure is as below. 1. LOGIN 2. NEEDAUTH - randomly generated 3. AUTH 4. LOGINOK - no value sent. O byte length 5. ERROR - can be 0 byte length or contain an error string 6. GETFILE 7. DATA - x bytes indicating size + x bytes of data Note: x can be 0. which means empty file! A Sample Messaging

C - Client; S - Server

C: LOGIN rajesh S: NEEDAUTH aa1123 C: AUTH <Hash(Password + aa1123)> S: LOGINOK C: GETFILE detail.txt S: ERROR "No Such file" C: GETFILE details.txt S: DATA 100 "some data ... of 100 bytes" Client closes the connection after receiving the DATA and stores the data received into a file.

Extras/Optional: (Attempting will lead to Extra credit)

Server side : 1. Server can timeout an idle connection, where the idle timeout can be set as a command line argument. 2. Server can close the connection if client sends 'x' number of bad GETFILE messages. 'x' can be made configurable. 3. Server side handling of BAD Clients sending BAD/Invalid Messages/Out-of-Sequence Messages eg. A Client connecting and trying to send GETFILE Message without sending LOGIN 4. Implementing a simple Server side log with timestamp. 5. Server support for parallel multiple client connections with select() and FDSET mechanism of checking socket data availability or truly parallel threaded server! -4- Client side: 1. Supporting Parallel Connections to download multiple files from the server at a same time. eg. Via the UI can get the list of files as input from the user and open multiple parallel connections to the server send it to the server and start downloading the files at the same time. - Will need to reuse the connections as well. 2. Implementing a simple Client side download log with timestamp. 3. Take a download folder as a command line argument and all downloaded files with unique names to be stored in this folder. eg. if same file is downloaded twice, each filename will be unique by suffixing it with a timestamp. Note: As can be seen, the filesize in this barebones mechanism is restricted to 232 bytes - 4GB as the length field is only 4 Bytes. Extras (needs both server and client side co-operative changes) - Ability to support better Authentication schemes. - Ability to support big files on server side and client side - Ability to support RANGE where the server can send a portion of the file.

RANGE message syntax:

RANGE start end --- Value field has delimiter as CRLF "\r\n" eg. RANGE details.txt 20 300 When server receives a RANGE message it reads the file only the range requested i.e. 20 to 300 bytes (both inclusive) and returns that data. so data size will be typically end-start+1 bytes in this case 300-20+1 = 281 bytes Of course, if the file is much smaller than 300 bytes, server can only send the available bytes. Server should not choke on invalid RANGE values sent by a client, if 'start' is greater than 'end', etc.

Comment by Niranjan early Monday morning, November 17th, 2014
is although here:http://wiki.netbsd.org/tutorials/tuning_netbsd_for_performance/
Comment by emmanuel-kasper Friday evening, November 7th, 2014

Hi,

I am interested in working on this project. I would like to know if any work is being done with respect to this project? Any response and pointers would be greatly appreciated!

Thanks, Vinay

Comment by Vinaykumar in the wee hours of Wednesday night, September 11th, 2014

Hi Radoslaw,

the GRex mystery is solved.

http://www.a1k.org/forum/showthread.php?p=763785

You have to use the first memorywindow (BAR) for the XROMBAR. After activating (here at $8000.0000 with a Radeon9250 256MB) you could copy the ROM to somwhere in the RAM. Next step, deactivate XROMBAR and reactivate the old BAR.

BR Andre "Ratte" Pfeiffer a1k.org

Comment by computerhobby Saturday afternoon, September 6th, 2014

It is not finished. Ask on tech-kern for more information about the current state.

Comment by dholland terribly early Friday morning, August 22nd, 2014
I was wondering if this project was finished or not as I am really interested in working on this project. Please reply to this comment as soon as possible.
Comment by Sanchit in the wee hours of Sunday night, July 28th, 2014
.. so keep well hydrated
Comment by spz early Sunday morning, July 20th, 2014

Please provide the source code the for the below question.

Message Details There are totally 7 messages - Syntax/Structure is as below. 1. LOGIN 2. NEEDAUTH - randomly generated 3. AUTH 4. LOGINOK - no value sent. O byte length 5. ERROR - can be 0 byte length or contain an error string 6. GETFILE 7. DATA - x bytes indicating size + x bytes of data Note: x can be 0. which means empty file! A Sample Messaging

C - Client; S - Server C: LOGIN rajesh S: NEEDAUTH aa1123 C: AUTH <Hash(Password + aa1123)> S: LOGINOK C: GETFILE detail.txt S: ERROR "No Such file" C: GETFILE details.txt S: DATA 100 "some data ... of 100 bytes" Client closes the connection after receiving the DATA and stores the data received into a file.

Comment by Saikrishna mid-morning Wednesday, July 9th, 2014

Using NetBSD based sockets, implement a TCP client and TCP Server to demonstrate trivially secure file download from server.

Environment/Tools required: Linux RHEL4/5/6 and gcc/g++ compiler and vi editor (recommended) OR Windows XP/Vista/7/8 with Visual studio.

Learning Objective for student:

  1. Understand Protocol messaging and parsing.
  2. Manage multiple connections at a server side
  3. Manage the state of a connection at both client and server side.
  4. Ability to use NetBSD Sockets via C API.
  5. Binary vs Text and practical issues with Endianness !
  6. Ability to effectively use fread, fwrite APIs for file management
  7. can also use read/write system calls.
  8. Understanding Standards and compatibility (and the inherent difficulty!) between different (not differing!) implementations. Note: This is a group assignment. Each group can have maximum of 10 students.
  9. If group of 10 students is opted, 5 students may contribute to the client and another 5 students may contribute to the server. but put together at a minimum, the expected result is a group's server should work with its own client.
  10. If a group's server/client are compatible with other group's client/servers, it will receive more credit. ** Each group is expected to write both the client and a server." -1- Details:

  1. The messaging mechanism is based on Type-Length-Value (TLV) The Message Format is TYPE Field ---- Length Field --- Value field --------------- ---- 8 bytes --- 4 Bytes int ---- buffer of length bytes ---- Note: If Length Field is ZERO, then the Value field is empty which is still a perfectly valid message.
  2. eg. LOGINOK message - See details below.
  3. All types are of size 8 chars and always uppercase. Valid Types for Client messages are: LOGIN, AUTH, GETFILE Valid Types for Server messages are: NEEDAUTH, LOGINOK, DATA, ERROR As Type field is 8 bytes, it will employ 'SPACE' as padding. i.e. LOGIN will be sent as "LOGIN "
  4. The message exchange details are as below: Client connects to the server. LOGIN is the first message sent by the client. If server requires authentication, it replies with NEEDAUTH message. If server does not require authentication, it replies with LOGINOK message. Client sends back AUTH message as a reply to NEEDAUTH message Server replies with LOGINOK if authentication successful, else replies will reply with ERROR and close the connection. Client if login was ok, i.e. received LOGINOK message, can send at any time, a file download request by sending the message GETFILE. Note: Only outgoing request must be in the network pipe. Upon receiving a GETFILE message, if server finds a matching file it can send the file contents back in the DATA message. If no matching file, it may send back an ERROR message.
  5. Note, any time the client can close the connection and go away,
  6. server needs to detect this and close the connection on its side as well. eg. if while server is sending a file, the client goes away, server needs to stop then and there. -2-
  7. The Client upon receiving the DATA message should store the contents appropriately into a file for later viewing.
  8. Server never advertises the file list on the server (unlike SCP/SFTP/FTP !)
  9. so clients need to know the file names
  10. a simple way is to query the user Note: A fancy UI is not required, rather a basic working UI with scanf() or fgets() should suffice. Note about GETFILE: The GETFILE message can send as value a full path name (absolute) or a relative path name. with "/" as directory separator. eg. If "hello/details.txt" is sent, server is supposed to look in its working directory the given path of the file. So in this case "hello/details.txt" and "details.txt" may refer to completely different files. Command Line arguments:

./TSServer ./TSClient Note about Auth Mechanism: Any 2-way auth mechanism can be employed, trivially a Salted Password hash based mechanism can be employed. This avoids replay attacks! NOTE: * If the environment chosen is Linux, it comes with a tool called 'md5sum' which can be used to generate the Hashes on both client side and server side. * The server can store the list of passwords in some file. (Bad! Not secure, but ok for this barebones version) * Both the server and client programs may run in the same machine during development, but for testing it would be ideal to run them in different machines, to see how a client/server reacts when network goes away. ( Intentionally pull out the ethernet cable when the client is downloading a big file!) -3-

Message Details

There are totally 7 messages - Syntax/Structure is as below. 1. LOGIN 2. NEEDAUTH - randomly generated 3. AUTH 4. LOGINOK - no value sent. O byte length 5. ERROR - can be 0 byte length or contain an error string 6. GETFILE 7. DATA - x bytes indicating size + x bytes of data Note: x can be 0. which means empty file! A Sample Messaging

C - Client; S - Server

C: LOGIN rajesh S: NEEDAUTH aa1123 C: AUTH <Hash(Password + aa1123)> S: LOGINOK C: GETFILE detail.txt S: ERROR "No Such file" C: GETFILE details.txt S: DATA 100 "some data ... of 100 bytes" Client closes the connection after receiving the DATA and stores the data received into a file.

Extras/Optional: (Attempting will lead to Extra credit)

Server side : 1. Server can timeout an idle connection, where the idle timeout can be set as a command line argument. 2. Server can close the connection if client sends 'x' number of bad GETFILE messages. 'x' can be made configurable. 3. Server side handling of BAD Clients sending BAD/Invalid Messages/Out-of-Sequence Messages eg. A Client connecting and trying to send GETFILE Message without sending LOGIN 4. Implementing a simple Server side log with timestamp. 5. Server support for parallel multiple client connections with select() and FDSET mechanism of checking socket data availability or truly parallel threaded server! -4- Client side: 1. Supporting Parallel Connections to download multiple files from the server at a same time. eg. Via the UI can get the list of files as input from the user and open multiple parallel connections to the server send it to the server and start downloading the files at the same time. - Will need to reuse the connections as well. 2. Implementing a simple Client side download log with timestamp. 3. Take a download folder as a command line argument and all downloaded files with unique names to be stored in this folder. eg. if same file is downloaded twice, each filename will be unique by suffixing it with a timestamp. Note: As can be seen, the filesize in this barebones mechanism is restricted to 232 bytes - 4GB as the length field is only 4 Bytes. Extras (needs both server and client side co-operative changes) - Ability to support better Authentication schemes. - Ability to support big files on server side and client side - Ability to support RANGE where the server can send a portion of the file.

RANGE message syntax:

RANGE start end --- Value field has delimiter as CRLF "\r\n" eg. RANGE details.txt 20 300 When server receives a RANGE message it reads the file only the range requested i.e. 20 to 300 bytes (both inclusive) and returns that data. so data size will be typically end-start+1 bytes in this case 300-20+1 = 281 bytes Of course, if the file is much smaller than 300 bytes, server can only send the available bytes. Server should not choke on invalid RANGE values sent by a client, if 'start' is greater than 'end', etc.

Comment by Ashish terribly early Tuesday morning, July 1st, 2014

Hello,

nice to see a more updated Xen how to.

But I am curious on that: why not use /usr/pkg/etc/rc.d instead of polluting /etc/rc.d with third part scripts?

Thank you, Raimundo Santos

Comment by Raimundo in the wee hours of Sunday night, April 28th, 2014
Add a comment