[Python-talk] A few more socket related questions

Fri Aug 28 14:53:44 EDT 2009

python-talk-bounces at dlslug.org wrote on 08/28/2009 02:08:58 PM:

> On Fri, Aug 28, 2009 at 12:05 PM, <bruce.labitt at autoliv.com> wrote:
> > Using TCP, How would one send a large array >> MTU ?
> 
>   Just write the data to the socket.  The TCP implementation handles
> the details.
> 
> > When it comes down to it, what is the real advantage of tcp?
> 
>   TCP is reliable, two-way, connection-oriented, and stream-oriented.
> TCP acts like a pipe: You put data in one end, it comes out the other,
> in the same order you put it in.  Actually a pair of pipes, flowing in
> opposite directions.
> 
>   UDP is unreliable, connectionless, one-way, unsequenced, and
> datagram-oriented.  UDP is more like a bunch of postcards, which might
> get delivered in any order, or not at all.
> 

Curious that tftp uses UDP.  My blade boots over NFS using tftp... 
Obviously there is a protocol superimposed over UDP that makes it reliable 
enough to send an OS.  So UDP is not all BAD.

> > The socket.send documentation still says to check the return value for 
the
> > number of bytes sent...  Is it just that tcp keeps trying to send?
> 
>   As I recall, for TCP, the send(2) call will block if you try to send
> more data than will fit in the transmit buffer, and won't return until
> TCP has consumed all the data you gave it.  I think UDP just returns
> an error.
> 

So using python, having an array that is 160MB, does one need to break it 
into pieces or not?  What you just wrote seems to be inconsistent with 
"just write the data to the socket".  I'm not saying this to be 
pointy-headed, I would like to understand this.  Actually, I NEED to 
understand this.

> > As I understand it, in tcp the client has to create and destroy a
> > socket(s) for each transaction.
> 
>   TCP and UDP do not recognize a concept of "transaction".

Bad choice of words.  Did not know how to describe the concept...

> 
>   For UDP, you create the socket, connect() it, and send data.  UDP
> has no idea if the destination host is even there.  I suppose you
> could call each datagram a "transaction", but UDP does not provide the
> semantics usually associated with the word "transaction".
> 

I guess by transaction, I'm including the higher protocol such as the 
receiver to acknowlege data reception.

>   For TCP, you create the socket and connect() it.  If the network
> subsystem can establish the connection with the remote host, the
> connect() succeeds.  You can then send/receive as much data as you
> want.  You can create more sockets to the same host, or just keep
> using the same one; that's up to you.
> 
>   If you want to impose transactions upon that TCP single stream, you
> can, but it's sometime you (or a library) has to do; it's not provided
> by TCP.  You could create, connect, and teardown a TCP connection for
> each transaction, but that's higher overhead. 

I would not want to tear down a socket unless I needed to...

> Most people either use
> a library, or just implement their own simply scheme.  For example,
> use an end-of-data marker.  That could be newline if it's all ASCII
> without formatting.  If it's arbitrary binary data, begin each
> transaction with a header that indicates the length.
> 

I would be sending arbitrary data, however, the first n bytes contain a 
command, the next m bytes would be either optional data for the command or 
data.  Uggh, as I write this it is clear I need to rethink this if I can 
just send the "array"

The socket would know what size buffer to use via a previously sencommand.

Optionally I could have a separate socket for every command?  If something 
came on that socket it means command x (implied) with data.  Don't fall 
over laughing, I'm thinking on the fly here.

>   There is a socket type for sequenced reliable packets, but I don't
> think IP normally supports it.
> 
> > How does the client know if all the data is sent?
> > How does the server know it got everything?
> 
>   TCP and UDP do not recognize a concept of "client" or "server".
> There are sender and receiver.
> 

I appreciate that.  However, I have to create both the client and server, 
and I need to know how to separate the components.  Right now I'm 
struggling with the jargon.  You can probably tell by the nature of the 
questions.  I'm straddling several worlds - systems engineering, 'classic 
programmming', networking,...  it is tough mapping the concepts through 
one world to the other. 

>   For UDP, the send() call is atomic, and will either send or return
> error.  As noted, you have no way of knowing if it made it to the
> receiver.  The receiver has to already be listening for it.
> 
>   For TCP, handshaking, flow control, acknowledgment and
> retransmission are built-in.  When the connection is set-up by the
> sender, it tries to contact the other end, which has to be listening
> for new TCP connections.  If not, the connect() will fail.  As I
> recall, once the TCP connection is up, the send() call may return an
> error if the network layer has detected a problem with the other end
> (timeout, excessive errors, etc.) and was blocking anyway.  If the
> call wasn't blocked, the network layer will send SIGPIPE to the
> process.
> 
> > Can the server send data to the client without a request?  (Delayed
> > output?)
> 
>   You'll have to clarify the above.  I suspect you're assuming
> semantics that TCP and UDP do not provide.

Indeed, this is really protocol level, isn't it?  (Or at least the next 
layer up.  jargon again!)

Client sends command to Server
                                                Server receives command, 
sends ack msg
client knows server got msg
                                                server sends 'busy' msg
client knows server is busy
                                                server sends results
client finally gets data

If the connection is still open, then it would appear this is easy to 
implement.

> 
> -- Ben

Thank you, Ben.  This conversation has been interesting!
-Bruce

******************************
Neither the footer nor anything else in this E-mail is intended to or constitutes an electronic signature and/or legally binding agreement in the absence of an express statement or Autoliv policy and/or procedure to the contrary. This E-mail and any attachments hereto are Autoliv property and may contain legally privileged, confidential and/or proprietary information. The recipient of this E-mail is prohibited from distributing, copying, forwarding or in any way disseminating any material contained within this E-mail without prior written permission from the author. If you receive this E-mail in error, please immediately notify the author and delete this E-mail. Autoliv disclaims all responsibility and liability for the consequences of any person who fails to abide by the terms herein. 
******************************