B4R Question SD card read and write slow when sd card inside have a 1000++ files

[tzfpg]

I use esp8266 to create ,read and write files to sd card when end user scan the rfid. (store user data)
when start all go smooth and great, but when files increase in sd card, I facing sd card read and write files speed slow down.

100++ using 0.9s
500++ using 2.2s
1000++ using 4.5s

my sd card files wills going increase, sd card speed slow down.

I just like to ask how I can improve the speed of read and write.

here is my read and write code:

Sub ReadSDData(filename As String) As String
    Dim result As String = ""
    If SD.OpenRead(filename) = True Then
        Dim NByte As Int=SD.Stream.BytesAvailable
        Dim Buffer(NByte) As Byte
        SD.Stream.ReadBytes(Buffer, 0, NByte)
        SD.Stream.Flush
        result = bc.StringFromBytes(Buffer)
        SD.Close
    End If
    Return result
End Sub

private Sub WriteSDData(filename As String,data As String)
    Dim Buffer() As Byte
    If SD.OpenReadWrite(filename) = True Then
        SD.Position = SD.CurrentFile.Size
        Buffer = bc.StringToBytes(data)
        SD.Stream.WriteBytes(Buffer,0,Buffer.Length)
        SD.Stream.Flush
        SD.Close
    End If
End Sub

 

[Erel]

Add some log methods to find which step is slow.

Make some tests. Put all other files in a different folder and see if it is faster when the number of files in the current folder is small.

 

[max123]

Add some log methods to find which step is slow.

Make some tests. Put all other files in a different folder and see if it is faster when the number of files in the current folder is small.

Hi,
I know this is an old post, but I've the same problem.

I read audio from SD and stream over UDP and sd slow down, do not matter the number of files, it is slow to read and write.

That it slow down if there are a lots of files I think it's normal, I had the same problem on Arduino IDE too... Sometime I use SD Card Formatter and it speed up a bit. Even the computer access it slow if there are a lots of files.

The UDP audio streaming on the Arduino IDE work well because in the SD.begin method I pass the speed 50000000 or SD_SCK_MHZ(50). If I do not pass it it slow down and there are a lots of audio clips. I do not remember the exact default speed inside SD.h library, but if you do not specify it, the default is very low speed like 4 Mhz or SPI_HALF_SPEED or SPI_QUARTER_SPEED.

So at this point I've some questions:

1) Because the rSD Initialize method do not provide (other that a CS pin) a second argument that set the SPI bus speed for internal SD library like Arduino ESP8266 Core and Arduino ESP32 Core do ?

2) I need to speed up readings, I suppose it use slow speed for SPI bus, or at least I want to change it manually, there is a way to change it ?

3) Maybe using inline C to do the 'SPI.setFrequency(freq)' after sd.Initialize method can do the trick ?

Many thanks for clearifications

 

[max123]

This is a begin function inside SD.h on ESP8266 Core, on ESP32 Core we optionally can pass SPI instance and even speed:

boolean begin(uint8_t csPin, uint32_t cfg = SPI_HALF_SPEED) {
    SDFS.setConfig(SDFSConfig(csPin, cfg));
        return (boolean)SDFS.begin();
    }