B4R Question ESP8266 - WebServer Headers Inssue

[Bangdido]

Hello, Thank you for this magnific example:

https://www.b4x.com/android/forum/threads/esp8266-wifi-remote-configuration.68596/#post-464800

I'm using a ESP-01, i'm trying to parse http headers from browser (for example to get a cookie) but the header are split,

this is an example of the header from browser (on my delphi web server and apache web server):

GET /settings HTTP/1.1
Host: 192.168.1.1
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:42.0) Gecko/20100101 Firefox/42.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: es-ES,es;q=0.8,en-US;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Cookie: session=6bOt36d0999T4da1e8214bc3a90328f50
Connection: keep-alive​

Using the same browser, i get it on esp-01 "Log(Buffer)" (using the same example and code of the demo above)

GET /settings HTTP/1.1
Host: 192.168.4.1
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:42.0) Gecko/20100101 Firefox/42.0
Accept-Lang
uage: es-ES,es;q=0.8,en-US;q=0.5,en;q=0.3
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate
Cookie: sessi
on=6bOt36d0999T4da1e8214bc3a90328f50
Connection: keep-alive

any clue about how to fix it ?
Thanks in advance

 

[Erel]

You should expect messages to be split or merged. This is how network communication works.

You can try to increase AsyncStreams.WaitForMoreData. The default value is 5. Increase it to 500. This will cause AsyncStreams to wait 500ms for more data.

A better solution will be to collect the data until you encounter a new line character and then parse it.

 

[Bangdido]

Increasing AsyncStreams.WaitForMoreData does not help,

any idea how to do that (collect data until new line)?
i'm having error here.

Sub Process_Globals
   Private FBuffer(1)  As String
end sub

Private Sub APStream_NewData (RawData() As Byte)
   FBuffer(0) = JoinStrings(Array As String(FBuffer(0), RawData(0)))
end sub

 

[Bangdido]

I'm having problems to understand strings and array of bytes..
i don't know how to use them.
Digging in the code i saw it on AsyncStreams.cpp Line #3
#define MAX_SIZE_NO_PREFIX 100

so increased to #define MAX_SIZE_NO_PREFIX 1000

and now i'm getting all the header

Length:328
[GET / HTTP/1.1
Host: 192.168.4.1
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:50.0) Gecko/20100101 Firefox/50.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: es-MX,es;q=0.7,en-US;q=0.3
Accept-Encoding: gzip, deflate
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1
]

 

[Erel]

I will make it configurable in the next version.

 

[Bangdido]

thank you, but you can help me with collecting data ?

i'm using a global String

Sub Process_Globals
   Private FBuffer(1)  As String
End Sub

Private Sub APStream_NewData(RawData() As Byte)
   FBuffer(0) = JoinStrings(Array As String(FBuffer(0),bc.StringFromBytes(RawData)))   
  Log("FBuffer:[",FBuffer(0),"]")
End Sub

but i get a crash everytime i ran it, i'm starting with arduino, and dont know how to use strings and bytes yet.

 

[Erel]

I'm not sure that I understand the question. What are you trying to do with the headers content?

 

[Bangdido]

Ok, let me try to explain,
i'm using a esp-01, it save data to eeprom memory (like ssid and password where to connect, access point name and password for local use)
firs time the module try to connect to ssid with password,
after 1 minute, if cant connect, switch to access point (AP) mode for configration

in AP open a webserver to dispatch compressed pages (static pages),
it have a login page where ask username and password to show/set configuration (ssid name and password, ap name and password)
that is where i'm trying to "parse" http headers, i'm using a POST method in login form to send username and password (i don't want to use GET
becose it send visible parameters in url)

if username and password are ok, i'll set a cookie on browser to navigate free on settings pages.
for some reason if i increase the length on AsyncStreams.cpp Line #3
#define MAX_SIZE_NO_PREFIX 100

i get the full header and i can process it, but when i use the POST method the esp-01 crashes (receiving the data)
and i dont know why.

AppStart
Setting configuration to default (first run)
Loading configuration from EEPROM memory
IP:192.168.0.105
Green are GET request, those working like charm.
Requesting / but is not logged, sending /login.html
Disconnected
Sending /css/style.css
Disconnected
Requesting / but is not logged, sending /login.html
Disconnected
Sending /css/style.css
Disconnected
Requesting / but is not logged, sending /login.html
Disconnected
Sending /css/style.css
Disconnected
Blue is a POST request, here just blow up, any advice ?
Header: POST /login HTTP/1.1
Header:Host: 192.168.0.105
Header:User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:50.0) Gecko/20100101 Firefox/50.0
Header:Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Header:Accept-Language: es-MX,es;q=0.7,en-US;q=0.3
Header:Accept-Encoding: gzip, deflate
Header:Referer: http://192.168.0.105/
Header: TNT: 1
Header:Connection: keep-alive
Header:Upgrade-Insecure-Requests: 1
there is no poison after the block. Expected poison address: 0x3fff188c, actual data: 0x8 0x85 0xfe 0x3f
block start: 3fff1634

Panic C:\Users\isramos\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.3.0\cores\esp8266\umm_malloc\umm_malloc.c:853 check_poison_block
ctx: cont
sp: 3ffef2f0 end: 3ffefc60 offset: 01b0
>>>stack>>>
3ffef4a0: 5ffffe00 5ffffe00 0000008b 40100334
3ffef4b0: 00000027 000011a8 3fff1628 4010044f
3ffef4c0: 00000076 0000003e 3ffeeb50 4010054f
3ffef4d0: 00000004 000005b4 00000624 401006da
3ffef4e0: 00000038 00000079 00000000 4010772c
3ffef4f0: 000000e0 00000015 00000096 402261e8
3ffef500: 000000b0 00000083 00000077 00000020
3ffef510: 00000005 00000048 0000004e 00000000
3ffef520: 00000000 3fff28f4 00000000 402275c0
3ffef530: 00000000 00000073 00000000 3ffee838
3ffef540: 00000073 00000000 00000000 40206ca1
3ffef550: 00000073 0000008b 00000000 00000067
3ffef560: 00000024 000000f5 3ffef58c 3ffefb84
3ffef570: 00000000 00000073 3fff2c4c 40206432
3ffef580: 00000000 000000e2 3ffee828 40206474
3ffef590: 00000000 000000f2 00000026 40207788
3ffef5a0: 3ffef600 3ffef5b0 3ffee828 4020270d
3ffef5b0: 3ffefa90 00000035 00000096 3ffefb7c
3ffef5c0: 3ffee808 000000e2 00000000 402075fe
3ffef5d0: 00000000 00000000 0000004a 0000007c
3ffef5e0: 000000e2 3fff1820 00000000 3ffee928
3ffef5f0: 3ffee958 000000e2 3ffee928 40201d5f
3ffef600: 3ffe8468 3ffe84e0 3ffefb84 40201d9a
3ffef610: 3ffef6b4 3ffefb7c 3ffefb84 40202efc
3ffef620: 3ffef6b4 00000070 0000007c 00000039
3ffef630: 3ffef6ac 3ffefb7c 3ffef6b4 3ffe84e0
3ffef640: 00000001 3ffe84e0 00000001 3ffe8548
3ffef650: 00000001 00000000 00000000 00000000
3ffef660: 00000000 00000000 00000000 3ffe84e0
3ffef670: 00000001 3ffe8508 00000001 3ffe84e0
3ffef680: 00000001 3ffe84f4 00000001 3ffe84e0
3ffef690: 3ffef601 3ffe84e4 00000001 3ffe84e0
3ffef6a0: 00000001 3ffef630 00000003 3ffe84d4
3ffef6b0: 00000001 3ffe84c8 00000001 3ffe84c4
3ffef6c0: ffffff01 3ffe84c0 00000001 00000017
3ffef6d0: 000001ec 3fff1820 3ffefb74 00000001
3ffef6e0: 00000000 00000000 3ffee928 000000f4
3ffef6f0: 000000c0 00000093 00000074 3ffefb7c
3ffef700: 3ffefb84 00000001 00000000 40203524
3ffef710: 00000008 00000000 00000000 00000000
3ffef720: 00000000 00000000 00000000 0000000b
3ffef730: 0000003c 00000090 0000003d 0000004f
3ffef740: 00000003 00000031 0000000c 00000086
3ffef750: 00000077 00000024 000000fe 00000083
3ffef760: 000000c9 0000005e 000000b2 00000032
3ffef770: 000000e4 00000022 00000055 00000050
3ffef780: 00000054 00000006 000000a0 00000012
3ffef790: 00000065 00000060 00000034 00000089
3ffef7a0: 0000004b 000000a2 0000003a 00000009
3ffef7b0: 0000004a 0000004c 000000a5 000000fe
3ffef7c0: 0000007b 00000072 000000d7 000000bb
3ffef7d0: 0000000e 00000096 0000001f 000000f9
3ffef7e0: 000000e3 0000007d 0000002d 0000009b
3ffef7f0: 000000bb 000000a7 000000f7 000000c7
3ffef800: 000000fd 000000d7 0000006e 00000003
3ffef810: 00000041 00000012 000000db 000000db
3ffef820: 0000001b 00000033 00000066 000000e8
3ffef830: 00000000 00000060 00000002 000000a1
3ffef840: 0000009f 000000a8 00000073 00000022
3ffef850: 00000041 00000070 00000001 0000006b
3ffef860: 00000023 00000019 000000d4 000000e7
3ffef870: 000000fe 00000079 000000f5 000000a0
3ffef880: 00000096 0000009e 00000044 00000061
3ffef890: 000000b2 0000002f 000000eb 0000006d
3ffef8a0: 00000049 00000004 0000006b 000000e7
3ffef8b0: 000000a8 00000015 00000040 00000086
3ffef8c0: 0000008f 000000d8 00000084 00000012
3ffef8d0: 000000c2 0000000a 0000005e 0000008b
3ffef8e0: 000000ff 000000e3 00000002 0000006f
3ffef8f0: 000000f1 00000044 0000006c 000000f4
3ffef900: 00000065 0000007c 000000de 000000e5
3ffef910: 00000098 0000008f 00000050 00000089
3ffef920: 00000007 000000d5 000000e4 000000cc
3ffef930: 000000d4 00000002 00000091 00000028
3ffef940: 00000008 00000095 0000000e 00000083
3ffef950: 00000072 000000ad 000000e9 000000a9
3ffef960: 0000007a 000000df 000000e9 000000e2
3ffef970: 00000066 000000f4 000000f5 00000028
3ffef980: 000000f3 0000005d 000000fc 00000079
3ffef990: 00000091 00000039 00000094 0000009a
3ffef9a0: 00000066 0000001e 0000003b 000000d5
3ffef9b0: 0000004e 000000e1 000000e3 00000009
3ffef9c0: 0000001c 00000063 0000006b 00000083
3ffef9d0: 0000000a 000000c4 000000bf 000000ca
3ffef9e0: 000000ee 000000f0 00000027 00000066
3ffef9f0: 00000084 0000005c 00000080 000000b2
3ffefa00: 0000002b 00000059 0000002a 0000007a
3ffefa10: 00000034 000000ba 0000008f 000000d9
3ffefa20: 00000065 00000067 000000d6 000000d3
3ffefa30: 00000057 00000041 000000a3 00000067
3ffefa40: 0000009f 0000006e 0000003d 000000be
3ffefa50: 000000e8 0000001f 00000000 00000000
3ffefa60: 000000ff 000000ff 00000003 00000000
3ffefa70: 000000b2 00000001 000000ee 000000b8
3ffefa80: 00000032 00000001 00000000 00000000
3ffefa90: 00088b1f 00000000 903c0b00 31034f3d
3ffefaa0: 2477860c 5ec983fe 22e432b2 06545055
3ffefab0: 606512a0 a24b8934 4c4a093a 727bfea5
3ffefac0: 960ebbd7 7de3f91f a7bb9b2d d7fdc7f7
3ffefad0: 1241036e 331bdbdb 6000e866 a89fa102
3ffefae0: 70412273 19236b01 79fee7d4 9e96a0f5
3ffefaf0: 2fb26144 04496deb 15a8e76b d88f8640
3ffefb00: 0ac21284 e3ff8b5e 44f16f02 7c65f46c
3ffefb10: 8f98e5de d5078950 02d4cce4 95082891
3ffefb20: ad72830e df7aa9e9 f466e2e9 5df328f5
3ffefb30: 399179fc 1e669a94 e14ed53b 631c09e3
3ffefb40: c40a836b f0eecabf 5c846627 592bb280
3ffefb50: ba347a2a 6765d98f 4157d3d6 6e9f67a3
3ffefb60: 1fe8be3d ffff0000 01b20003 0132b8ee
3ffefb70: 3ffe0000 3ffe8568 3fff1801 3ffe84c4
3ffefb80: 3ffe8401 3ffefa90 402000e2 3ffeec40
3ffefb90: 000001ec 3fff1828 049f12c9 00000000
3ffefba0: 40105a34 049f12c9 3ffeeab4 000001da
3ffefbb0: 3ffee784 3ffee7c0 3fff1634 4020493b
3ffefbc0: 3ffe8588 00000101 3ffe8584 40207701
3ffefbd0: 402010ae 00000005 3ffee928 000001da
3ffefbe0: 3ffee784 3fff1634 3ffee928 40201d1c
3ffefbf0: 00000000 00000000 000003e8 402077d0
3ffefc00: 00000000 3fff2c4c 40207340 3ffeec30
3ffefc10: 3fffdad0 3fff1614 3ffee7a8 40202ae8
3ffefc20: 3fffdad0 3ffee7b0 00000000 40202b4f
3ffefc30: feefeffe 00000000 3ffeec28 4020586b
3ffefc40: feefeffe feefeffe 3ffeec28 40207394
3ffefc50: feefeffe feefeffe 3ffeec40 40100970
<<<stack<<<
ets Jan 8 2013,rst cause:2, boot mode1,6)
ets Jan 8 2013,rst cause:4, boot mode1,6)
wdt reset

 

[Erel]

You shouldn't use JoinStrings or bc.StringFromBytes. They require a lot of memory and are not needed for parsing the content.

Check this tutorial: https://www.b4x.com/android/forum/threads/strings-and-bytes.66729/#content