[Python-talk] A few more socket related questions

[Ric Werme]

> Curious that tftp uses UDP.  My blade boots over NFS using tftp... 
> Obviously there is a protocol superimposed over UDP that makes it reliable 
> enough to send an OS.  So UDP is not all BAD.

UDP is very simple for a kernel or boot ROM to implement,  TCP has a lot of state
to deal with and funky issues with how acknowledgements are sent or might be sent.
Tftp booting generally is done over a LAN, so few messages get lost and dumb
retransmit code timeouts work well.

For your situation, all the TCP hassles are handled by the OS, and all you deal
with is a byte stream.

> > > The socket.send documentation still says to check the return value for 
> the
> > > number of bytes sent...  Is it just that tcp keeps trying to send?

The only time there will be a difference is when some error occurs, typically
the receiver closes the connection, less often the network path is broken and
connectivity is lost.  You won't see it in normal operation.

[Ben wrote:]
> >   As I recall, for TCP, the send(2) call will block if you try to send
> > more data than will fit in the transmit buffer, and won't return until
> > TCP has consumed all the data you gave it.  I think UDP just returns
> > an error.

send() will return on error or when the last of the user data is copied into
the kernel.  It doesn't mean that the data has been sent or has reached the
destination.  Don't worry about it.

UDP will return an error if the sender's kernel buffers overflow.  It's your
responsibility to devise a protocol to prevent that.  Worse, the receiver
side will throw away incoming packets if its kernel buffers are full and
you receive no notification.  (Well, maybe via ICMP, but don't expect it.)
It's your responsibility to devise a protocol to prevent receiver overflow too.
The receiver application receives no indication of data loss.

> So using python, having an array that is 160MB, does one need to break it 
> into pieces or not?

Not with TCP.  Send it as a big chunk or many big chunks, the kernel will
deal with creating TCP packets, retransmission, reordering, and some
congestion control.  With UDP, send pieces no bigger than the
MTU size _including_ UDP, IP, and ethernet protocol headers.  That might be 8,
20, and 14 bytes, but the IP size varies with implementations.  Use TCP.

  What you just wrote seems to be inconsistent with 
> "just write the data to the socket".  I'm not saying this to be 
> pointy-headed, I would like to understand this.  Actually, I NEED to 
> understand this.

With TCP, just write the data to the socket.  The OS does the rest.
Use TCP.

> >   There is a socket type for sequenced reliable packets, but I don't
> > think IP normally supports it.

Technically, it's not an IP issue, it another protocol, a peer of UDP and TCP.

> > > How does the client know if all the data is sent?

I think when you close the connection the OS might wait for everything to
go out.  Don't worry about it, if you sent it to the kernel, the kernel
will get it to the receiver unless something odd happens or you abort
the connection.

> > > How does the server know it got everything?

When you close the connection, the program on the server will read the last
of the data and then return an EOF condition (i.e. read() returns 0 bytes),
just like reading from a file.  With UDP, it's your responsibility to devise
a protocol that lets the server know when all the data has arrived.  Use TCP,
don't worry about closing a connection to mark end of data.

[I'ts not that easy]  If you want to use a connection to send FFT data and
receive the results, and then close the connection, you need the kernel
shutdown system call to shut down the client's send side.  There's likely
something in the socket library for that.  Still, use TCP.

> >   TCP and UDP do not recognize a concept of "client" or "server".
> > There are sender and receiver.

A TCP connection can send and receive, a UDP "connection" can send and receive.

> I appreciate that.  However, I have to create both the client and server, 
> and I need to know how to separate the components.  Right now I'm 
> struggling with the jargon.  You can probably tell by the nature of the 
> questions.

Typically a server is something that is waiting for a request or will start running
when a request arrives, a client is something that initiates a request.

Go to Starbucks, you're the client, the barrista is the server.