B4J Question Problems with HttpJob in API-REST GET

[Nator]

Hello Community, it is possible that you can help me determine what could be the problem when consulting a REST API Get the code is the following

ApiRquest.Initialize("estaciones", Me)
    
    ApiRquest.Download("https://api.captahydro.com/devices")
    
    ApiRquest.GetRequest.SetHeader("APIKey", "4c823e19-e47a-yyyy-yyyy-xxxxxxxxxxxx")
    ApiRquest.GetRequest.SetHeader("OrganizationId", "29")
    ApiRquest.GetRequest.SetHeader("Content-Type","application/json")

the error returned is the following :

ResponseError. Reason: , Response: {"Message":"User is not authorized to access this resource with an explicit deny"}


when making the query in postman I have no problems and the records are returned without problems

I also tried download2 and it doesn't work either, I would really appreciate your valuable help

 

[Nator]

Unless you, for some reason, are changing the 3455 from the B4J app to 8837 in Postman, the displayed portion of the API keys do not match, therefore you may not be using the same key for B4J as for Postman. Please make sure you're actually using the same API keys (right now it looks like you are not). Alternative, you could copy the B4J key (if they differ) into Postman and check the result(s)

I changed the characters to not show the real ones, but the apikey for b4j and postman have always been correct

 

[Nator]

yeah, 3455 is wrong. i can run a shell in b4j and get the output using 8837

I changed the characters to not show the real ones, but the apikey for b4j and postman have always been correct

 

[drgottjr]

forget about the key; you already showed it. nobody here cares. it is, however, much easier to get to the bottom of the problem with it. if you're worried, just get a new one.
you can obtain the information you're looking for with my suggestion. did you try it? did you even look at it? i've spent a lot of time trying to help, you know.
it does not use okhttputils2, but it will give you the answer.
there is a problem with ssl. i see the error, but researching it and a solution is time-consuming.

amazon aws is issuing the error. if you break one of its rules (IAM), there is a punishment as spelled out in its configuration.
in this case, the punishment is an "explicit deny". below please find some debugging output from okhttputils2 and from curl.

note, first of all, that the b4j project was compiled with the accept all option.
you will see the http error (403) and some more details relating to the error.

regarding curl, if you read each line slowly, you will notice 3 references to (20) during the ssl handshake.
the last reference to (20) defines what error 20 means: SSL certificate verify result: unable to get local issuer certificate (20)

the compiler option HU2_ACCEPTALL essentially says we (the client) don't care if a server's ssl certificate is somehow invalid.
what it apparently does not control is what happens if amazon is not happy with some aspect of the ssl handshake.

people ask, "why does something work with postman but not with b4x?" those people are not alone. there are numerous references
on stack overflow to similar complaints (having nothing to do with b4x).

my solution involving b4j's running curl in a shell will provide the json the op wanted. as far as i am concerned, it's a viable solution.
i cannot at present see why okhttptuils2 does not work (either in b4j or b4a). trying to load the url in a webview also fails, but for reasons
relating to android and cors.

with luck, the following output might provide some inspiration:

FROM OKHTTPUTILS2:
(Http client initialized with accept all option.)
ResponseError. Reason: , Response: {"Message":"User is not authorized to access this resource with an explicit deny"}
403
{"Message":"User is not authorized to access this resource with an explicit deny"}
(MyMap) {content-length=[82], content-type=[application/json], date=[Wed, 12 Apr 2023 22:41:00 GMT], x-amz-apigw-id=[DST7gFaqvHcFZvg=], x-amzn-errortype=[AccessDeniedException], x-amzn-requestid=[8967c286-a81d-4baa-9b60-2c3f1fe2957f]}

FROM CURL:
C:\Users\new>curl -k -v -H "APIKey: 4c823e19-e47a-493f-8837-ade4efea9dfc" -H "OrganizationId: 29" https://api.captahydro.com/devices/
* Hostname was NOT found in DNS cache
*   Trying 35.83.58.134...
* Connected to api.captahydro.com (35.83.58.134) port 443 (#0)
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using ECDHE-RSA-AES128-GCM-SHA256
* Server certificate:
*        subject: CN=*.captahydro.com
*        start date: 2023-02-07 00:00:00 GMT
*        expire date: 2023-07-20 23:59:59 GMT
*        issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M01
*        SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> GET /devices/ HTTP/1.1
> User-Agent: curl/7.35.0
> Host: api.captahydro.com
> Accept: */*
> APIKey: 4c823e19-e47a-493f-8837-ade4efea9dfc
> OrganizationId: 29
>
< HTTP/1.1 200 OK
< Date: Wed, 12 Apr 2023 18:16:21 GMT
< Content-Type: application/json
< Content-Length: 1879
< Connection: keep-alive
< x-amzn-RequestId: 9207a630-3868-41b8-8f28-e4513bbb31f2
< x-amz-apigw-id: DRtJrFkEPHcFasA=
< X-Amzn-Trace-Id: Root=1-6436f570-5f0114830fcf6e5971ae0020;Sampled=0;lineage=21
1b84fa:0|ff6a7589:0
<
[{"id":"43","name":"Canal El Olivar","battery_soc":85,"battery_soc_measured":168
1323067000,"flow":1222.7928010000005,"flow_measured":1681323088000,"level":17.90
0000000000006,"level_measured":1681323088000},{"id":"virtual-9983bfa1-9afe-40d9-
b038-2e9f4e9793c2","name":"Ext 1"},{"id":"virtual-e89621e3-4bf6-4f29-a31b-d4c147
36b4cd","name":"Ext 2 y 3 Las Quemadas","flow":0.050991727270215115,"flow_measur
ed":1650893502000,"level":2.5,"level_measured":1650893502000},{"id":"virtual-d1e
71930-6476-4ca7-9bfc-4330a66a7bb4","name":"Ext 4 El Alamo","level":2.3,"level_me
asured":1650892712000,"flow":0.0401134110990327,"flow_measured":1650892712000},{
"id":"virtual-6a1166d0-4288-46e4-a064-01d63a246b68","name":"Ext 5 Willer","level
":2.4,"level_measured":1650287326000,"flow":0.016770412484647616,"flow_measured"
:1650287326000},{"id":"virtual-34c74275-287c-4d76-b831-61eff0fbc8a9","name":"Ext
 6 El Carmen","flow":0.03383479023135801,"flow_measured":1650891642000,"level":2
.3,"level_measured":1650891642000},{"id":"virtual-643821c2-b49a-4788-9a9d-17a645
f56e14","name":"Ext 7 La Mirandina","level":2.2,"level_measured":1650891349000,"
flow":0.031652335142924294,"flow_measured":1650891349000},{"id":"virtual-8c40a09
8-da86-4c7a-a0cb-2d239792b3cf","name":"Ext 8 Las Nu+¦ez","flow":0.02951893443198
7887,"flow_measured":1650889896000,"level":2.1,"level_measured":1650889896000},{
"id":"virtual-d4e00cc0-ecf0-4768-91b8-6f5b0ce189fb","name":"Ext 9 El Peral","lev
el":2.4,"level_measured":1650369727000,"flow":0.03011631850010887,"flow_measured
":1650369727000},{"id":"virtual-20adb717-6f8e-4f1b-b163-ff365c5f73d8","name":"Ex
t 11 El Olivo","level":2,"level_measured":1650555096000,"flow":0,"flow_measured"
:1650555096000},{"id":"107","name":"Colihues Bajo","flow":336.878775,"flow_measu
red":1681323066000,"level":50.5,"level_measured":1681323066000,"battery_soc":100
,"battery_soc_measured":1681323034000}]* Connection #0 to host api.captahydro.co
m left intact

 

[OliverA]

with luck, the following output might provide some inspiration:

SSLv3, TLS handshake, Client hello (1):

Maybe this?

* **`COMPATIBLE_TLS` no longer supports SSLv3.** In response to the
[POODLE](http://googleonlinesecurity.blogspot.ca/2014/10/this-poodle-bites-exploiting-ssl-30.html)
vulnerability, OkHttp no longer offers SSLv3 when negotiation an
HTTPS connection. If you continue to need to connect to webservers
running SSLv3, you must manually configure your own `ConnectionSpec`.

Source: https://android.googlesource.com/platform/external/okhttp/+show/master/CHANGELOG.md#341

 

[Nator]

forget about the key; you already showed it. nobody here cares. it is, however, much easier to get to the bottom of the problem with it. if you're worried, just get a new one.
you can obtain the information you're looking for with my suggestion. did you try it? did you even look at it? i've spent a lot of time trying to help, you know.
it does not use okhttputils2, but it will give you the answer.
there is a problem with ssl. i see the error, but researching it and a solution is time-consuming.

amazon aws is issuing the error. if you break one of its rules (IAM), there is a punishment as spelled out in its configuration.
in this case, the punishment is an "explicit deny". below please find some debugging output from okhttputils2 and from curl.

note, first of all, that the b4j project was compiled with the accept all option.
you will see the http error (403) and some more details relating to the error.

regarding curl, if you read each line slowly, you will notice 3 references to (20) during the ssl handshake.
the last reference to (20) defines what error 20 means: SSL certificate verify result: unable to get local issuer certificate (20)

the compiler option HU2_ACCEPTALL essentially says we (the client) don't care if a server's ssl certificate is somehow invalid.
what it apparently does not control is what happens if amazon is not happy with some aspect of the ssl handshake.

people ask, "why does something work with postman but not with b4x?" those people are not alone. there are numerous references
on stack overflow to similar complaints (having nothing to do with b4x).

my solution involving b4j's running curl in a shell will provide the json the op wanted. as far as i am concerned, it's a viable solution.
i cannot at present see why okhttptuils2 does not work (either in b4j or b4a). trying to load the url in a webview also fails, but for reasons
relating to android and cors.

with luck, the following output might provide some inspiration:

FROM OKHTTPUTILS2:
(Http client initialized with accept all option.)
ResponseError. Reason: , Response: {"Message":"User is not authorized to access this resource with an explicit deny"}
403
{"Message":"User is not authorized to access this resource with an explicit deny"}
(MyMap) {content-length=[82], content-type=[application/json], date=[Wed, 12 Apr 2023 22:41:00 GMT], x-amz-apigw-id=[DST7gFaqvHcFZvg=], x-amzn-errortype=[AccessDeniedException], x-amzn-requestid=[8967c286-a81d-4baa-9b60-2c3f1fe2957f]}

FROM CURL:
C:\Users\new>curl -k -v -H "APIKey: 4c823e19-e47a-493f-8837-ade4efea9dfc" -H "OrganizationId: 29" https://api.captahydro.com/devices/
* Hostname was NOT found in DNS cache
*   Trying 35.83.58.134...
* Connected to api.captahydro.com (35.83.58.134) port 443 (#0)
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using ECDHE-RSA-AES128-GCM-SHA256
* Server certificate:
*        subject: CN=*.captahydro.com
*        start date: 2023-02-07 00:00:00 GMT
*        expire date: 2023-07-20 23:59:59 GMT
*        issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M01
*        SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.
> GET /devices/ HTTP/1.1
> User-Agent: curl/7.35.0
> Host: api.captahydro.com
> Accept: */*
> APIKey: 4c823e19-e47a-493f-8837-ade4efea9dfc
> OrganizationId: 29
>
< HTTP/1.1 200 OK
< Date: Wed, 12 Apr 2023 18:16:21 GMT
< Content-Type: application/json
< Content-Length: 1879
< Connection: keep-alive
< x-amzn-RequestId: 9207a630-3868-41b8-8f28-e4513bbb31f2
< x-amz-apigw-id: DRtJrFkEPHcFasA=
< X-Amzn-Trace-Id: Root=1-6436f570-5f0114830fcf6e5971ae0020;Sampled=0;lineage=21
1b84fa:0|ff6a7589:0
<
[{"id":"43","name":"Canal El Olivar","battery_soc":85,"battery_soc_measured":168
1323067000,"flow":1222.7928010000005,"flow_measured":1681323088000,"level":17.90
0000000000006,"level_measured":1681323088000},{"id":"virtual-9983bfa1-9afe-40d9-
b038-2e9f4e9793c2","name":"Ext 1"},{"id":"virtual-e89621e3-4bf6-4f29-a31b-d4c147
36b4cd","name":"Ext 2 y 3 Las Quemadas","flow":0.050991727270215115,"flow_measur
ed":1650893502000,"level":2.5,"level_measured":1650893502000},{"id":"virtual-d1e
71930-6476-4ca7-9bfc-4330a66a7bb4","name":"Ext 4 El Alamo","level":2.3,"level_me
asured":1650892712000,"flow":0.0401134110990327,"flow_measured":1650892712000},{
"id":"virtual-6a1166d0-4288-46e4-a064-01d63a246b68","name":"Ext 5 Willer","level
":2.4,"level_measured":1650287326000,"flow":0.016770412484647616,"flow_measured"
:1650287326000},{"id":"virtual-34c74275-287c-4d76-b831-61eff0fbc8a9","name":"Ext
 6 El Carmen","flow":0.03383479023135801,"flow_measured":1650891642000,"level":2
.3,"level_measured":1650891642000},{"id":"virtual-643821c2-b49a-4788-9a9d-17a645
f56e14","name":"Ext 7 La Mirandina","level":2.2,"level_measured":1650891349000,"
flow":0.031652335142924294,"flow_measured":1650891349000},{"id":"virtual-8c40a09
8-da86-4c7a-a0cb-2d239792b3cf","name":"Ext 8 Las Nu+¦ez","flow":0.02951893443198
7887,"flow_measured":1650889896000,"level":2.1,"level_measured":1650889896000},{
"id":"virtual-d4e00cc0-ecf0-4768-91b8-6f5b0ce189fb","name":"Ext 9 El Peral","lev
el":2.4,"level_measured":1650369727000,"flow":0.03011631850010887,"flow_measured
":1650369727000},{"id":"virtual-20adb717-6f8e-4f1b-b163-ff365c5f73d8","name":"Ex
t 11 El Olivo","level":2,"level_measured":1650555096000,"flow":0,"flow_measured"
:1650555096000},{"id":"107","name":"Colihues Bajo","flow":336.878775,"flow_measu
red":1681323066000,"level":50.5,"level_measured":1681323066000,"battery_soc":100
,"battery_soc_measured":1681323034000}]* Connection #0 to host api.captahydro.co
m left intact

First I want to thank you for your help, and second I already tried it on windows 10 and it works, the problem is that this app is running on linux ubuntu 20.04, I tried the curl instruction and it returns the known error of denial

root@developer:/# curl -k -v -H "APIKey: 4c823e19-e47a-493f-8837-ade4efea9dfc" -                                                                                                             H "OrganizationId: 29" https://api.captahydro.com/devices/
*   Trying 35.167.120.69:443...
* TCP_NODELAY set
* Connected to api.captahydro.com (35.167.120.69) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=*.captahydro.com
*  start date: Feb  7 00:00:00 2023 GMT
*  expire date: Jul 20 23:59:59 2023 GMT
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M01
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x5557a4f0f320)
> GET /devices/ HTTP/2
> Host: api.captahydro.com
> user-agent: curl/7.68.0
> accept: */*
> apikey: 4c823e19-e47a-493f-8837-ade4efea9dfc
> organizationid: 29
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 403
< date: Thu, 13 Apr 2023 14:07:14 GMT
< content-type: application/json
< content-length: 82
< x-amzn-requestid: 860d1bb0-d6ad-46d6-b43e-de6ef20e767f
< x-amzn-errortype: AccessDeniedException
< x-amz-apigw-id: DUbm-GKRvHcF6qQ=
<
* Connection #0 to host api.captahydro.com left intact
{"Message":"User is not authorized to access this resource with an explicit deny}

 

[aeric]

Not sure this help. I read an answer from SO that you need to deploy your API.

 

[drgottjr]

you may have ended up on a different server, so they could be configured differently. but they're both run by amazon.
your call: Connected to api.captahydro.com (35.167.120.69) port 443 (#0)
mine: Connected to api.captahydro.com (35.83.58.134) port 443 (#0)
your system uses cloudfare dns, mine was using google dns.
anyway, result is the same, but the tls/ssl handshake is totally different which is what makes me thing there is another server involved.

(running stuff as root is not recommended. not talking about security, but the system behaves in particular ways when root is active.)

add these 3 switches to curl
--http1.1 --no-alpn --no-npn
run

assuming failure, you got 9 permutations;. work your way through them one at a time: 1 + 2, 1, 1 + 3, 2 + 3, etc etc keep notes, keep logs. good luck

************************** UPDATE ********************
so i was trying the permutations myself on my freebsd box. i got curl to work doing this:

edit your /etc/hosts file to use 35.83.58.134 as the ip address for api.captahydro.com. if you haven't done this before, su to root and vi /etc/hosts:
35.83.58.134 api.captahydro.com

this should bypass dns lookup, which on your system might use a different ip address.

then just add --http1.1 to the curl command. (without --no-alpn --no-npm).

my freebsd box is in a completely foreign country (with no extradition treaty to my country), so there is no connection to my desktop pc. i needed both changes (as described above), but it worked. tested several times.

 

[drgottjr]

i can do it in b4j with okhttputils2. see attached. you don't seem to be very interested anymore, so i'll leave it at that. if you want to know how to do it, post to the thread.

 

[OliverA]

if you want to know how to do it, post to the thread.

I want to know and so does anyone else that finds this thread with similar problems! (Sort of what this forum is for - a resource of answers to issues encountered. Don't leave us hanging!!!)

Running this code on my machine

Dim jo As JavaObject = Me
jo.RunMethod("doSetHttpVersion", Array(HttpUtils2Service.hc))

Results in

java.lang.RuntimeException: Method: doSetHttpVersion not found in: b4j.example.b4xmainpage

Btw, if you could post your code in the appropriate code tags and outputs in the appropriate quote tags that would be really appreciated! And thank you for all your efforts in figuring this out.

 

[drgottjr]

i know how to post code. i know "this will help others with the same problem, etc". i am at present pissed with the op. i want some engagement on his/her part. he could have voluntarily told me one suggested solution worked for him. i had to ask him if he had even bothered to try it. i realize he may have other worries in his life, but it's clear a number of people here on the forum are spending time with what i assume is something related to his job. i'm free to get annoyed. when i calm down, i'll do the right thing for all the other forum members who are experiencing this particular coding problem.

 

[OliverA]

i'm free to get annoyed

Understood. It looks like I need some coffee myself since

Dim jo As JavaObject = Me
jo.RunMethod("doSetHttpVersion", Array(HttpUtils2Service.hc))

It just means you got some inline Java code.
Interested in seeing what your solution is.

 

[Nator]

i know how to post code. i know "this will help others with the same problem, etc". i am at present pissed with the op. i want some engagement on his/her part. he could have voluntarily told me one suggested solution worked for him. i had to ask him if he had even bothered to try it. i realize he may have other worries in his life, but it's clear a number of people here on the forum are spending time with what i assume is something related to his job. i'm free to get annoyed. when i calm down, i'll do the right thing for all the other forum members who are experiencing this particular coding problem.

I thank you again for your disinterested help, I tried the previous code in linux and there was no case that it worked for me (it may be that I am not very good with linux), I will try this new one that you have posted

 

[Nator]

i can do it in b4j with okhttputils2. see attached. you don't seem to be very interested anymore, so i'll leave it at that. if you want to know how to do it, post to the thread.

This error shows me "Error Description: HttpUtils2 Service is declared twice. You must remove the reference to the library or code module."

I have the two referenced libraries HttpUtils2 and OkHttpUtils2 if I remove any of the two the code throws an error

 

[drgottjr]

attached find the project.

 

[Nator]

attached find the project.

oh great now it works thanks for your huge help, I just have one question about the compiler option HU2_PUBLIC, what does it really mean?, thank you very much

 

[drgottjr]

don't forget to mark the thread as "solved". that way i can still be considered for the tesla prize at the end of the year.

as to HU2_PUBLIC, it allows the okhttp client to be public instead of private. normally you cannot see it outside of the service. and in order to see it, the service has to be running. normally, the service doesn't start to run until you actually start the job. but that's too late, so you have to start the service earlier when it's initializing. that way you can access the public client.

for whatever reason, there is some conflict between your server and okhttp and amazon's ssl rules which occurs during the ssl handshake and connection stage. maybe just in the connection phase, i'm not sure. unfortunately, the error messages were not very helptul. after testing a number of things, it seemed that using http/1.1 was the only thing that made amazon happy. normally, we do not have to worry about it, but in this case i had to force okhttp to use http/1.1. in this case.

any change in the server or amazon's rule set or a new okhttp library could nullify my fix.

 

[Nator]

don't forget to mark the thread as "solved". that way i can still be considered for the tesla prize at the end of the year.

as to HU2_PUBLIC, it allows the okhttp client to be public instead of private. normally you cannot see it outside of the service. and in order to see it, the service has to be running. normally, the service doesn't start to run until you actually start the job. but that's too late, so you have to start the service earlier when it's initializing. that way you can access the public client.

for whatever reason, there is some conflict between your server and okhttp and amazon's ssl rules which occurs during the ssl handshake and connection stage. maybe just in the connection phase, i'm not sure. unfortunately, the error messages were not very helptul. after testing a number of things, it seemed that using http/1.1 was the only thing that made amazon happy. normally, we do not have to worry about it, but in this case i had to force okhttp to use http/1.1. in this case.

any change in the server or amazon's rule set or a new okhttp library could nullify my fix.

Thank you very much for your appreciated help and sharing your knowledge of this fabulous b4x tool.