+- Atozed Forums (https://www.atozed.com/forums)
+-- Forum: Indy (https://www.atozed.com/forums/forum-8.html)
+--- Forum: Indy General Discussion (https://www.atozed.com/forums/forum-9.html)
+--- Thread: How can know length of readstring in Idtcpserver (/thread-2496.html)
How can know length of readstring in Idtcpserver - Zsanvs - 08-27-2021
Hello，
We have 4G IOT adapter which can send json package to my delphi idtcpserver ， i use
Memo1.Lines.Add('receiveString-> ：' + AContext.Connection.IOHandler.Readstring(160));
but length of string is not fixed, like following 4 json packages, length of third one is different with others, so i can't fix in IOHandler.Readstring(???)).
{"devId":"492C201010012591","msgType":"aiValueRpt","data":{"AI1":"0","AI2":"0","AI3":"0","AI4":"0"},"timestamp":"1630043052"}
{"devId":"492C201010012591","msgType":"aiValueRpt","data":{"AI1":"0","AI2":"0","AI3":"0","AI4":"0"},"timestamp":"1630043053"}
{"devId":"492C201010012591","msgType":"rs485ValueRpt","data":"0103040ADC1786B643","timestamp":"1630043053"}
{"devId":"492C201010012591","msgType":"aiValueRpt","data":{"AI1":"0","AI2":"0","AI3":"0","AI4":"0"},"timestamp":"1630043054"}
How can read these strings by idtcpserver?
Thank you!
Ike
RE: How can know length of readstring in Idtcpserver - rlebeau - 08-27-2021
(08-27-2021, 05:50 AM)Zsanvs Wrote: How can read these strings by idtcpserver?
Since the strings are not fixed length, ReadString() is not appropriate for this task, unless the client sends the string length before the string data, which does not appear to be the case in your example.
Are the JSON messages delimited, such as with a line break? If so, then you can use ReadLn() instead.
If not, then the best way to handle this would be to use a JSON engine that supports push parsing. Then you can read arbitrary amounts of raw bytes from the socket, such as with ReadBytes(-1), and then push those bytes into the engine, and let it spit out complete JSON messages as they become available.
If that is not an option, then you will just have to buffer the bytes yourself as it arrives, and then parse only complete messages in the buffer. For instance, in this case, each JSON message starts with a {, so keep buffering until the closing } is received (keeping track of nested {} pairs in the middle), and then parse the buffer up to that closing }, leaving any remaining bytes in the buffer for subsequent parsing of the next message.
It is really difficult to give you a definitive answer without knowing the exact protocol your IOT device is actually using on the wire.
RE: How can know length of readstring in Idtcpserver - Zsanvs - 08-30-2021
(08-27-2021, 04:25 PM)rlebeau Wrote:(08-27-2021, 05:50 AM)Zsanvs Wrote: How can read these strings by idtcpserver?
Since the strings are not fixed length, ReadString() is not appropriate for this task, unless the client sends the string length before the string data, which does not appear to be the case in your example.
Are the JSON messages delimited, such as with a line break? If so, then you can use ReadLn() instead.
If not, then the best way to handle this would be to use a JSON engine that supports push parsing. Then you can read arbitrary amounts of raw bytes from the socket, such as with ReadBytes(-1), and then push those bytes into the engine, and let it spit out complete JSON messages as they become available.
If that is not an option, then you will just have to buffer the bytes yourself as it arrives, and then parse only complete messages in the buffer. For instance, in this case, each JSON message starts with a {, so keep buffering until the closing } is received (keeping track of nested {} pairs in the middle), and then parse the buffer up to that closing }, leaving any remaining bytes in the buffer for subsequent parsing of the next message.
It is really difficult to give you a definitive answer without knowing the exact protocol your IOT device is actually using on the wire.
Thank you for your reply.
I found each JSON message start with {"devId":"492C201010012591" ，is it possible to split message by use ReadLn() ?
or as you mentioned another option buffer function, please give me some coding for your solution if it's possible .
it's hard for me to find some demo online.Thank you for your help.
RE: How can know length of readstring in Idtcpserver - Zsanvs - 08-30-2021
(08-30-2021, 01:32 AM)Zsanvs Wrote:(08-27-2021, 04:25 PM)rlebeau Wrote:(08-27-2021, 05:50 AM)Zsanvs Wrote: How can read these strings by idtcpserver?
Since the strings are not fixed length, ReadString() is not appropriate for this task, unless the client sends the string length before the string data, which does not appear to be the case in your example.
Are the JSON messages delimited, such as with a line break? If so, then you can use ReadLn() instead.
If not, then the best way to handle this would be to use a JSON engine that supports push parsing. Then you can read arbitrary amounts of raw bytes from the socket, such as with ReadBytes(-1), and then push those bytes into the engine, and let it spit out complete JSON messages as they become available.
If that is not an option, then you will just have to buffer the bytes yourself as it arrives, and then parse only complete messages in the buffer. For instance, in this case, each JSON message starts with a {, so keep buffering until the closing } is received (keeping track of nested {} pairs in the middle), and then parse the buffer up to that closing }, leaving any remaining bytes in the buffer for subsequent parsing of the next message.
It is really difficult to give you a definitive answer without knowing the exact protocol your IOT device is actually using on the wire.
Thank you for your reply.
I found each JSON message start with {"devId":"492C201010012591" ，is it possible to split message by use ReadLn() ?
or as you mentioned another option buffer function, please give me some coding for your solution if it's possible .
Thank you for your help.
Thank you for your option of Buffer, I think i have found solution :
Code:
while not AContext.Connection.IOHandler.InputBufferIsEmpty() do
begin
str:=str+AContext.Connection.IOHandler.ReadString(1) ;
end;
if not str.IsEmpty then
Memo1.lines.Add('ReceiveString-> ：' +str);It's working well moment.
Thank you again.
RE: How can know length of readstring in Idtcpserver - rlebeau - 08-30-2021
(08-30-2021, 01:32 AM)Zsanvs Wrote: I found each JSON message start with {"devId":"492C201010012591" ，is it possible to split message by use ReadLn() ?
Since that is the start of each message, not the end, I would use that string with TIdIOHandler.WaitFor() instead. But that won't tell you where each message ends. You should not have to wait for the beginning of the next message before processing the previous message.
(08-30-2021, 01:32 AM)Zsanvs Wrote: or as you mentioned another option buffer function, please give me some coding for your solution if it's possible .
See further below.
(08-30-2021, 02:51 AM)Zsanvs Wrote: Thank you for your option of Buffer, I think i have found solution :
That is not a good solution, for many reasons:
- If the IOHandler.InputBuffer is empty, you are not reading more bytes into it. You stop reading when there are no more bytes in the buffer. But what if the device sends a lot of messages in a short time? In any case, you could have replaced that entire loop with a single call to IOHandler.InputBufferAsString() instead.
- Not that it matters, because this code still doesn't try to detect the actual end of each message properly.
- IOHandler.ReadString() reads and decodes the specified number of bytes into Characters. You are reading only 1 byte at a time, not 1 Character at a time. So that will work only for single-byte Characters, but will fail on multi-byte characters, like non-ASCII characters encoded in UTF-8. If you really want to read 1 Character at a time, use IOHandler.ReadChar() instead of IOHandler.ReadString(). But it will still fail on Unicode characters > U+FFFF.
- This code runs in a worker thread, so you MUST synchronize with the main UI thread when touching UI controls, like your TMemo.
So, again I ask you, what is the actual protocol that the IOT device is using on the wire? Do you have any documentation for that? If not, then can you at least capture a session containing multiple messages? Either with a packet sniffer like Wireshark, or by assigning one of Indy's TIdLog... components to the IOHandler.Intercept property.
In the meantime, try something more like this:
Code:
var
Level, BufSize, StartPos, I: Integer;
str: string;
...
AContext.Connection.IOHandler.WaitFor('{"devId":', False);
BufSize := AContext.Connection.IOHandler.InputBuffer.Size;
StartPos := 1;
Level := 1;
repeat
while (StartPos < BufSize) and (Level > 0) do
begin
case AContext.Connection.IOHandler.InputBuffer.PeekByte(StartPos) of
Ord('{'): Inc(Level);
Ord('}'): Dec(Level);
end;
Inc(StartPos);
end;
if Level = 0 then Break;
AContext.Connection.IOHandler.CheckForDataOnSource(100);
AContext.Connection.IOHandler.CheckForDisconnect(True, True);
until False;
str := AContext.Connection.IOHandler.ReadString(StartPos);
TThread.Synchronize(nil,
procedure
begin
Memo1.lines.Add('ReceiveString-> ：' +str);
end
);