Wish Looking for a way to get compiler not to put string constants into RAM

BertI · Feb 27, 2020

As a result of some recent issues I learnt the hard way that the compiler for some reason puts literal strings (constants) into RAM even though these are immutable anyway. This seems like a waste of valuable resource even on something like an ESP8266. In searching around this topic there are numerous references to Progmem and something called the 'F' macro which on the face of it appear to provide a way to get the compiler not to put such strings into RAM. As my familiarity in this area is very poor I have no idea whether this is something that could be somehow implemented generically or via inline C within B4R, whether it would be different for ESP8266 as compared to Arduino (as implicated by the link below), or even if it is meaningful at all. However one way or another I think it would be very useful to have a user friendly way to avoid clogging up RAM for constants. One of many examples talking about the issue: https://arduino-esp8266.readthedocs.io/en/latest/PROGMEM.html

Erel · Feb 27, 2020

Using PROGMEM is demonstrated here: https://www.b4x.com/android/forum/threads/using-progmem-to-save-memory.71114/#content
I'm not sure whether it will work with ESP8266.

BertI · Feb 28, 2020

Erel said:
Using PROGMEM is demonstrated here: https://www.b4x.com/android/forum/threads/using-progmem-to-save-memory.71114/#content
I'm not sure whether it will work with ESP8266.

Yes I've looked at this several times but can't figure out how to use this information for this context. So one might typically have the following situation:
Astream.Write("string 1")
Astream.Write("string 2")
etc

and ideally would like some function which conceptually might be written as:
Astream.Write(FromFlash("string 1"))
Astream.Write(FromFlash("string 2"))
etc

where FromFlash is a way to tell the compiler that the following string is to be held only in flash

For example seems in C you might achieve this type of thing as per this example:
void setup() {
Serial.begin(115200); Serial.println();
Serial.println( F("This is an inline string")); //
Serial.printf_P( PSTR("This is an inline string using printf %s"), "hello");
}

You'll have to excuse me if this is complete nonsense, but one conceptual way to be able to use the F or PSTR functions might be to somehow cast the whole Astream.Write statements into the block of inline C but, as I say, I don't know if such a thing is somehow possible (meaning open stream in B4R and then somehow write to this within the C.)

BertI · Mar 2, 2020

Ok so perhaps not so elegant but the following is a possible way to use the PROGMEM functionality for a page of strings which does work on an ESP8266:

B4X:

#Region Project Attributes
    #AutoFlushLogs: True
    #CheckArrayBounds: True
    #StackBufferSize: 2000
#End Region

Sub Process_Globals
    Public Serial1 As Serial
    Public pgmstrlen As UInt
    Public pagelen As UInt
End Sub

Private Sub AppStart
    Serial1.Initialize(115200)
    Log("AppStart")
    Log("RAM: ", AvailableRAM)
    getstring("page1")
    Log("RAM afterwards: ", AvailableRAM)
    getstring("page2")
    Log("RAM afterwards: ", AvailableRAM)
End Sub

Private Sub getstring(page As String)
    GetByte(page,0)
    Dim str(pgmstrlen) As Byte
    Log("Str len = ",pgmstrlen)
    For i = 0 To pgmstrlen-1
        str(i)=GetByte(page,i)
    Next
    Log(str)
End Sub

Private Sub GetByte(page As String, Index As UInt) As Byte
    Select page
        Case "page1"
            Return RunNative("getpage1", Index)
        Case "page2"
            Return RunNative("getpage2", Index)
    End Select
End Sub

#if C
#include <avr/pgmspace.h>
const char page1[] PROGMEM = { //change the data here
"Some kind of text line stored in flash instead of SRAM\n\
and yet another line of text\n\
and blah blah blah blah blah blah blah blah blah blah blah blah\n\
and blah blah blah blah blah blah blah blah blah blah blah blah\n\
and final line of text"
};

const char page2[] PROGMEM = { //change the data here
"Some other text blah2 blah2 bla2h blah2 blah2 blah2 blah2 blah2\n\
blah2 blah2 bla2h blah2 blah2 blah2 blah2 blah2\n\
blah2 blah2 bla2h blah2 blah2 blah2 blah2 blah2\n\
blah2 blah2 bla2h blah2 blah2 blah2 blah2 blah2\n\
blah2 blah2 bla2h blah2 blah2 blah2 blah2 blah2\n\
and final line of blah2"
};

B4R::Object beo1;
B4R::Object* getpage1(B4R::Object* o) {
    b4r_main::_pgmstrlen = sizeof(page1);  
   return beo1.wrapNumber(pgm_read_byte_near(page1 + o->toLong()));
};

B4R::Object* getpage2(B4R::Object* o) {
    b4r_main::_pgmstrlen = sizeof(page2);  
   return beo1.wrapNumber(pgm_read_byte_near(page2 + o->toLong()));
};
#end if

In the above example I am deliberately retrieving the data byte by byte into a local array as the RAM space for this will be released after the getstring sub is exited. However for streaming this data out to a web page I'd probably not even fill an array first so that minimal RAM overhead is needed and I can then reduce StackBufferSize - or at least that's the theory, haven't tried yet.

Now I have another one of my dumb questions... Is there any way that I can reference the character arrays in the inline C from B4R such that I don't have to have a separate object to call for each page (as in the example above)? In other words conceptually pass the name of the string array that I want as a parameter in the RunNative call and then just one sub in the inline C which then uses this to reference the required array. I can use a global variable to pass the index as my understanding is that only one parameter can be passed via the RunNative call.

Thanks to tigrot I have seen this post https://www.b4x.com/android/forum/t...ation-and-lookup-in-progmem.88640/post-560823, which maybe has some clues of how to achieve something indirectly but I want to avoid having to copy the string data into temporary arrays and also to avoid having to maintain a table of array names in the inline C if possible. I suppose it will probably be possible to copy one byte at a time with a modified version of this example (to my present level of understanding) but just seems like another convoluted turn if there was a simpler way to directly reference the array names.

miker2069 · Apr 29, 2020

This is just rough, but might spark an idea. You define you discrete pages as you've done:

C:

const char page0[] PROGMEM = { //change the data here
"Some kind of text line stored in flash instead of SRAM\n\
and yet another line of text\n\
and blah blah blah blah blah blah blah blah blah blah blah blah\n\
and blah blah blah blah blah blah blah blah blah blah blah blah\n\
and final line of text"
};

const char page1[] PROGMEM = { //change the data here
"Some kind of text line stored in flash instead of SRAM\n\
and yet another line of text\n\
and blah blah blah blah blah blah blah blah blah blah blah blah\n\
and blah blah blah blah blah blah blah blah blah blah blah blah\n\
and final line of text"
};

You can then define an array of char * pointers to those pages, call it "pages":

C:

#define MAX_PAGES  2
const char *pages[MAX_PAGES]={page0,page1};

And then a small change to your getpage function:

C:

B4R::Object* getpage(B4R::Object* o) {
 
    uint8_t page_idx = b4r_main::_pageidx; //you'll need another process global variable for the page idx
    uint16_t byte_idx = o->toLong();
 
    b4r_main::_pgmstrlen = sizeof(page1);
 
   return beo1.wrapNumber(pgm_read_byte_near(&pages[page_idx] + byte_idx));
};

Now your B4R GetByte Sub could look like this:

B4X:

Private Sub GetByte(PageIndex_arg As byte, Index As UInt) As Byte
    'now all you need is one call to getpage
 
    PageIndex = PageIndex_arg ' PageIndex is a process global that will be referenced in the inline routine
    'could add some page index checking here
    Return RunNative("getpage", Index)
 
End Sub

By the way - the byte by byte seems awfully expensive time-wise. Since you've allocated an array in a B4R sub that ulimately will call down into inline c, you could just pass the byte array pointer to your inline c and copy all in one call using strcpy_P (The strcpy_P() function is similar to strcpy() except that src is a pointer to a string in program space. ). Something like this in c:

C:

 strcpy_P(buffer, (char*)pgm_read_word(&(pages[page_idx])));

buffer points to your allocated byte array in B4R . One and Done...

BertI · Apr 29, 2020

Thank you for this explanation. I had seen something which had some aspects of similarity here: https://www.b4x.com/android/forum/t...ation-and-lookup-in-progmem.88640/post-560823 , but still some bits which either I didn't understand fully or perhaps didn't understand how to utilise effectively for my particular requirement. In part probably because of my near zero fluency in C. Whilst I understand the concepts of pointers and such, deciphering what the syntax is doing still gets me... so additional expalnations always help to consolidate what I think I know or more often - don't.

Some questions:
In the 3rd code section, line 6: b4r_main::_pgmstrlen = sizeof(page1);
Presume this might be just a leftover from my code as 'page1' is directly referenced in the C here rather than passed in some way from B4R. So in this case to actually get the size of an indirectly referenced array via page_idx would I have to somehow count the bytes in C? Or maybe this would be unecessary anayway if I wasn't trying to get byte by byte

In the B4X code section line 4 , should PageIndex = PageIndex_arg actually be pageidx = PageIndex_arg? I.e a typo, or am I missing something?

Yes I think you are right about the inefficiency of the byte by byte transfer, but although my example showed allocation of an array I was actually thinking to send the data byte by byte to Astream.write, rather than filling an array and then sending that. However I think there is some folly in that thought since sending bytes to astream.write fills a buffer array anyway.

Wish Looking for a way to get compiler not to put string constants into RAM

BertI

Member

Erel

B4X founder

BertI

Member

BertI

Member

miker2069

Active Member

BertI

Member

Similar Threads

Wish Looking for a way to get compiler not to put string constants into RAM

BertI

Member

Erel

B4X founder

BertI

Member

BertI

Member

miker2069

Active Member

BertI

Member

Similar Threads

Privacy & Transparency

Privacy & Transparency