How to send POST data to a webserver?

[rlebeau]

I have made a copy of the logfile and using the HxD hex editor I have cut away all data preceding the start of the file transfer (where the magic byte E9 is located) and following the arrival of the last byte in the file.
Then I have used WinMerge to compare the original bin file with the edited version of the logfile.
What I find is 9 instances where the following byte pattern has been inserted, making up for the additional 81 bytes:

Code:

0D 0A 53 65 6E 74 20 3A 20

In plain text: "<CRLF>Sent : "  (not including the quotes)

Then you didn't strip off everything that the log had added. The file get uploaded in chunks, what you are seeing is status text being written to the log file for each of those chunks. That text is not transmitted to the server, though.

Notice that these addresses are exactly $8009 (32777 decimal) bytes apart!
The inserted bytes are 9 so it seems like after every $8000 bytes sent the extra stuff is inserted.

Sounds about right.

Now the question is:
Are these items actually sent or are they artifacts of the TIdLogFile logging?

They are artifacts of the logging, they are not transmitted to the server.

If you switch from TIdLogFile to TIdLogStream and use a TFileStream for output, you won't see that status text written to the log file anymore.

If I could find a way to export the file body content from a Wireshark logfile, then it would be possible to compare the data there. But I am totally lost in Wireshark due to the number of commands and options etc.

Wireshark is really not that hard to use. And there are TONS of tutorials on it.

In this case, once you have filtered the HTTP stream, you can select the POST message, and in the Packet Details pane, right-click on the file data and choose "Exported Selected Packet Bytes".

Just being able to export exactly the bytes sent by the PC to the IoT device would make it possible to then use a hex editor to find the start of the file transfer and verify if there is a difference between the FireFox and Indy transfers.

That is exactly the type of thing that Wireshark is good for.

It looks like the only difference is that Indy adds

Code:

Content-Transfer-Encoding: binary

just before the file data whereas FireFox does not.

As I already commented earlier, that should not affect the upload. But I did show you exactly how to remove that header if necessary.

Also, there is another difference - FireFox is using HTTP 1.1, whereas TIdHTTP is using HTTP 1.0 instead. That also should not affect the upload, but just in case, if you want TIdHTTP to use HTTP 1.1 then set the TIdHTTP.ProtocolVersion property to pv1_1, and enable the hoKeepOrigProtocol flag in the TIdHTTP.HTTPOptions property.

I tried to find a way to block the Content-Transfer-Encoding item from being produced but failed....

As I showed you on May 3rd:

Code:

Src.AddFile('update', FileName).ContentTransfer := '';

Following the POST command, what is the correct way to retrieve the return string from the server.

Your IoT device is not returning a proper HTTP error response code on failure, so you will have to parse the returned HTML instead. There are multiple overloaded versions of TIdHTTP.Post() - some that return the response body as a String, and some that fill an output TStream with the response body. Use one that returns a String, and then parse it as needed:

Code:

var
  Response: String;

try
  Response := HTTP.Post(URL, Src);  //Send the update file
  // parse Response as needed...
  Result := ...;
except
  // post error. Not happening in your case, but you should handle it nonetheless...
  Result := False;
end