B4J Question [SOLVED] Altered order when generating json from a map

mc73 · Sep 5, 2023

Hi, I'm generating a json from a map, however the final json though absolutely valid, loses the order of the items either in or out an object. This can be a bit problematic if a user decides to manually edit the json file, so it would be nice to have a solution. I will implement a function to re-organise the json, however in the mean time I'm really curious about why this may be happening.

Can anyone with a better vision on this, present the cause?

Here's the log of the original map:

Map:

(MyMap) {Description=test description, Input Port=BMT 1, Output Port=BMT 1, Sequences=[{Description=test desc, Active=Yes, Channel=0, Message Type=CC, Control Number=10, Value From=0, Value To=127, Diff From=0, Diff To=1, Operator=Or, Steps=[{Type=Keystrokes, Active=Yes, Keys=t e s t }]}]}

And here's the json generated by the JSONGenerator:

JSON:

{
  "Sequences": [
    {
      "Operator": "Or",
      "Steps": [
        {
          "Type": "Keystrokes",
          "Active": "Yes",
          "Keys": "t e s t "
        }
      ],
      "Active": "Yes",
      "Diff From": "0",
      "Description": "test desc",
      "Value To": "127",
      "Diff To": "1",
      "Channel": "0",
      "Value From": "0",
      "Control Number": "10",
      "Message Type": "CC"
    }
  ],
  "Description": "test description",
  "Input Port": "BMT 1",
  "Output Port": "BMT 1"
}

walterf25 · Sep 5, 2023

mc73 said:
Hi, I'm generating a json from a map, however the final json though absolutely valid, loses the order of the items either in or out an object. This can be a bit problematic if a user decides to manually edit the json file, so it would be nice to have a solution. I will implement a function to re-organise the json, however in the mean time I'm really curious about why this may be happening.

Can anyone with a better vision on this, present the cause?

Here's the log of the original map:

Map:

(MyMap) {Description=test description, Input Port=BMT 1, Output Port=BMT 1, Sequences=[{Description=test desc, Active=Yes, Channel=0, Message Type=CC, Control Number=10, Value From=0, Value To=127, Diff From=0, Diff To=1, Operator=Or, Steps=[{Type=Keystrokes, Active=Yes, Keys=t e s t }]}]}

And here's the json generated by the JSONGenerator:

JSON:

{ "Sequences": [ { "Operator": "Or", "Steps": [ { "Type": "Keystrokes", "Active": "Yes", "Keys": "t e s t " } ], "Active": "Yes", "Diff From": "0", "Description": "test desc", "Value To": "127", "Diff To": "1", "Channel": "0", "Value From": "0", "Control Number": "10", "Message Type": "CC" } ], "Description": "test description", "Input Port": "BMT 1", "Output Port": "BMT 1" }

Take a look at this post from Erel here

mc73 · Sep 5, 2023

walterf25 said:
Take a look at this post from Erel here

Thank you very much Walter.
I don't like inserting another level of complexity in the json, so either I'll leave it as is, or try a manual generator. I will update if I come up with a good one

mc73 · Sep 5, 2023

So, I've made a custom generator, it works for me, however I haven't thoroughly tested it.

Custom JSON Generator for keeping the order of objects:

Sub customJson(myMap As Map) As String
    Return appendMap(myMap)
End Sub

Sub appendMap(myMap As Map) As String
    Private json As StringBuilder
    json.Initialize
    json.Append("{"&CRLF)
    For Each key As String In myMap.Keys
        Private myObject As Object=myMap.Get(key)
        json.Append(appendObject(key,myObject))
    Next
    Private jsonString As String=json.ToString
    If jsonString.EndsWith(","&CRLF) Then
        jsonString=jsonString.SubString2(0,jsonString.Length-2)
    End If
    jsonString=jsonString&CRLF&"}"
    Return jsonString
End Sub

Sub appendList(myList As List) As String
    Private json As StringBuilder
    json.Initialize
    json.Append("["&CRLF)
    For Each myObject As Object In myList
        json.Append(appendObject("",myObject))
    Next
    Private jsonString As String=json.ToString
    If jsonString.EndsWith(","&CRLF) Then
        jsonString=jsonString.SubString2(0,jsonString.Length-2)
    End If
    jsonString=jsonString&CRLF&"]"
    Return jsonString
End Sub

Sub appendObject(key As String,myObject As Object) As String
    Private json As StringBuilder
    json.Initialize
    If key.Length>0 Then
        json.Append(escapeJSONChars(key)&": ")
    End If
    If     myObject Is List Then
        Private lst As List=myObject
        json.Append(appendList(lst))
    Else if myObject Is Map Then
        Private anotherMap As Map=myObject
        json.Append(appendMap(anotherMap))
    Else
        json.Append(escapeJSONChars(myObject))
    End If
    json.Append(","&CRLF)
    Return json.ToString
End Sub

Sub escapeJSONChars (text As String) As String
    Return Chr(34)&text.Replace("\","\\").Replace(Chr(34),"\"&Chr(34)).Replace(CRLF,"\n").Replace(Chr(13),"\r").Replace(TAB,"\t").Replace("/","\/")&Chr(34)
End Sub

EnriqueGonzalez · Sep 5, 2023

Hey don't think your code guarantees that it will save in the correct order (for maps). May be try with an orderedmap

[B4X] B4XOrderedMap - get first item, nth item and last item

B4XOrderedMap from B4XCollections, is a combination of a List with a Map. Avoid calling OrderedMap.Values unless you need to access all values, as it creates and fills a new List each call. The Keys list, on the other hand, is a list maintained by the collection and can be accessed directly...

www.b4x.com

tchart · Sep 6, 2023

You can use this if you have a single level Map e.g. a config file.

B4X:

Sub AppStart (Args() As String)   
    Dim Config As Map
    Config.Initialize
    
    Config.Put("audit_web",True)
    Config.Put("ts_host","localhost")
    Config.Put("web_ignore_ext",".js|.css|.aspx")
    Config.Put("commas","1,2,3")
    Config.Put("audit_ptl", True)
    Config.Put("ts_port", 9090)   
    
    Log(MapToSortedJSONString(Config))
End Sub


Sub MapToSortedJSONString(Input As Map) As String   
    Dim Cleaned As String = Input.As(JSON).ToString
    
    Cleaned = Cleaned.SubString2(1,Cleaned.Length-1) ' Remove braces
    Cleaned = Cleaned.Trim 'Trim any empty space
    
    Dim Values As List
    Values.Initialize
    
    For Each Value As String In Regex.split(","&Chr(10),Cleaned) ' (JSON).ToString uses Chr(10) for new lines and not CRLF
        Values.Add(Value.Trim) ' Trim each line
    Next
    
    Values.Sort(True)
        
    Dim Glue As String = ","&CRLF
    Dim Indent As String = "    "
    
    Dim sb As StringBuilder
    sb.Initialize
    For Each s As String In Values
        sb.Append(Indent).Append(s).Append(Glue)
    Next
    If sb.Length > 0 Then sb.Remove(sb.Length - Glue.Length, sb.Length)
    
    Return $"{${CRLF}${sb.ToString}${CRLF}}"$
End Sub

mc73 · Sep 6, 2023

EnriqueGonzalez said:
Hey don't think your code guarantees that it will save in the correct order (for maps). May be try with an orderedmap

[B4X] B4XOrderedMap - get first item, nth item and last item

B4XOrderedMap from B4XCollections, is a combination of a List with a Map. Avoid calling OrderedMap.Values unless you need to access all values, as it creates and fills a new List each call. The Keys list, on the other hand, is a list maintained by the collection and can be accessed directly...

www.b4x.com

Hi, in this particular project the forEach in map.keys works well without the need of a b4xOrdersMap, but yes, it's true that if one tries to use this code in a more general project it may fail.

mc73 · Sep 6, 2023

tchart said:

You can use this if you have a single level Map e.g. a config file.

B4X:

Sub AppStart (Args() As String)  
    Dim Config As Map
    Config.Initialize
   
    Config.Put("audit_web",True)
    Config.Put("ts_host","localhost")
    Config.Put("web_ignore_ext",".js|.css|.aspx")
    Config.Put("commas","1,2,3")
    Config.Put("audit_ptl", True)
    Config.Put("ts_port", 9090)  
   
    Log(MapToSortedJSONString(Config))
End Sub


Sub MapToSortedJSONString(Input As Map) As String  
    Dim Cleaned As String = Input.As(JSON).ToString
   
    Cleaned = Cleaned.SubString2(1,Cleaned.Length-1) ' Remove braces
    Cleaned = Cleaned.Trim 'Trim any empty space
   
    Dim Values As List
    Values.Initialize
   
    For Each Value As String In Regex.split(","&Chr(10),Cleaned) ' (JSON).ToString uses Chr(10) for new lines and not CRLF
        Values.Add(Value.Trim) ' Trim each line
    Next
   
    Values.Sort(True)
       
    Dim Glue As String = ","&CRLF
    Dim Indent As String = "    "
   
    Dim sb As StringBuilder
    sb.Initialize
    For Each s As String In Values
        sb.Append(Indent).Append(s).Append(Glue)
    Next
    If sb.Length > 0 Then sb.Remove(sb.Length - Glue.Length, sb.Length)
   
    Return $"{${CRLF}${sb.ToString}${CRLF}}"$
End Sub

Hi, thank you. Unfortunately the map I work on has more levels inside.

mc73 · Sep 6, 2023

tchart said:
(JSON).ToString uses Chr(10) for new lines and not CRLF

Note that in B4X CRLF is in fact the chr(10).

tchart · Sep 6, 2023

mc73 said:
Note that in B4X CRLF is in fact the chr(10).

Thanks for pointing this out, I had never noticed this.

I always assumed it was Chr(13) + Chr(10) since the name is CRLF

The reason I usually replace Chr(10) with Chr(13) & Chr(10) is that some versions of Notepad on Windows wont show the new lines correctly.

I never noticed this (as I use Notepad++) until a customer pointed out that it was difficult to update a config file as it was all on one line.

B4X:

ConfigString = ConfigString.Replace(Chr(10), Chr(13) & Chr(10))

B4J Question [SOLVED] Altered order when generating json from a map

mc73

Well-Known Member

walterf25

Expert

mc73

Well-Known Member

mc73

Well-Known Member

EnriqueGonzalez

Well-Known Member

[B4X] B4XOrderedMap - get first item, nth item and last item

tchart

Well-Known Member

mc73

Well-Known Member

[B4X] B4XOrderedMap - get first item, nth item and last item

mc73

Well-Known Member

mc73

Well-Known Member

tchart

Well-Known Member

Similar Threads