B4XDatabase

[dw_b4x]

I am interested to see the results of this project.

I know there is Cloud KVS. But being able to sync databases would be the ideal app. For example having a database on a mobile device that can work offline then when they return to the office they can press one button and both databases would be synced. In an ideal world there would be logging so that all the changes can be viewed by date,time and user. In Europe there are standards regarding the deletion of data from a database.

I think it is important to have utilities to reindex,repair,export the databases.

You should also explain the database structure. How the information is saved in the file.

How many users would the server support .

It would also be interesting to see 'real world' test results.

 

[sfsameer]

Awesome. My question would be on offline and online sync functionality.

I am using couchdb for offline web apps which have a functionality to sync when online. Off course this is a a nosql approach, no possibility of running sql queries or joins as it's merely like key value pairs. However will an awesome rest api, they say it's like Mongo.

Also, the dB engine you will release will only work on vps environments and not shared hosting?

Hello,
1- It will contain Offline-Online data sync
2- It can work on windows server, VPS, or a free hosting (apache) for example 000webhost

 

[Indic Software]

99% of the database engines are out-dated, and they rarely add new features, and top 2 database in the world (MS SQL and Oracle) cost millions per license.

You forgot to mention PostgreSQL . It is free, open source and has almost all features that Oracle has to offer and much much more like Java Objects.

 

[Indic Software]

Our aim is to create a database engine that can be faster, easier, more secure than any database engine out there, imagine you have 1 million records stored on your hard disk or your server with a size of only 500KB and the select query would take 2-3 milliseconds, which database engine is able to do that?

This is a very tall claim. Storing a million rows in 500 kb!?!?!!!

As far as I know to achieve this you will have to compress the data at time of writing and uncompress it after reading it. This will surely take up a lot of CPU cycles.

 

[Indic Software]

I know a local manufacturer that pays 1.9 Million dollars per year for Oracle just because they use their database, and i could swear that the performance is slower than internet explorer

The performance part is surprising as Oracle is the leader in RDBMS. I am saying this because we have a SaaS based web app for managing education institutions developed in PHP and which uses MySQL.

There are 67 institutions using our solution and the number of rows in some of the tables are more then 8 million and it is working without any problem.

But still the very ideas of having an alternate RDBMS available itself is elating. Keep it up. I would surely test your RDBMS in due course.

 

[aaronk]

How much is this going to be ?
What server specs are required to run this ?
Is it going to run on Linux servers (non-ui app) or only plan to release a desktop version ?

 

[sfsameer]

How much is this going to be ?
What server specs are required to run this ?
Is it going to run on Linux servers (non-ui app) or only plan to release a desktop version ?

There will be 2 versions, a Free version and a cloud version
Once it's released we will announce the features in both versions

Hello,
2- It can work on windows server, VPS, or a free hosting (apache) for example 000webhost

And a cloud version (Web App)

 

[Sandman]

99% of the database engines are out-dated, and they rarely add new features

That's two fascinating claims you made there, without providing any actual facts for neither. I'm going to assume you're not really talking about discontinued database solutions but rather something like to top twenty ones? So at least 19 of them are outdated? I realize it's difficult to have deep knowledge on all databases, but what would you say are the most common ways they are outdated?

And you say they rarely add new features. Looking at Oracle, MySQL, MS SQL and PostgreSQL, they've been around for 25-40 years and could on many aspects be considered feature complete. Is it a bad thing? Do you think they should keep on adding features regardless what, just so they can't be criticized on forums?

top 2 database in the world (MS SQL and Oracle) cost millions per license.

Top 2 by what criteria? Price? According to the list i link below, which is sorted by popularity and seem very updated (Wikipedia also references it, so I'm going to assume it's a respected site), we have MySQL on spot #2, well above MS SQL. MySQL was bought by Oracle a couple of years ago, which caused the original developers to fork it into MariaDB, which still is free. And is on place 12 on the list. Just to take an example of a popular database that is free. And let's not forget about PostgreSQL which many consider an absolute piece of art when it comes to databases.

It's an interesting list really: https://db-engines.com/en/ranking

Our aim is to create a database engine that can be faster, easier, more secure than any database engine out there, imagine you have 1 million records stored on your hard disk or your server with a size of only 500KB and the select query would take 2-3 milliseconds, which database engine is able to do that?

That's an ambitious goal and I really look forward to seeing your database on their site. Don't forget to register here so we can see how your database compares to the other 381 systems currently ranked: https://db-engines.com/en/user_login

 

[sfsameer]

That's an ambitious goal and I really look forward to seeing your database on their site. Don't forget to register here so we can see how your database compares to the other 381 systems currently ranked: https://db-engines.com/en/user_login

Will do.

 

[josejad]

which makes you stand closer to Erel (in my humble opinion).

do not blaspheme!

 

[sfsameer]

That's two fascinating claims you made there, without providing any actual facts for neither. I'm going to assume you're not really talking about discontinued database solutions but rather something like to top twenty ones? So at least 19 of them are outdated? I realize it's difficult to have deep knowledge on all databases, but what would you say are the most common ways they are outdated?

And you say they rarely add new features. Looking at Oracle, MySQL, MS SQL and PostgreSQL, they've been around for 25-40 years and could on many aspects be considered feature complete. Is it a bad thing? Do you think they should keep on adding features regardless what, just so they can't be criticized on forums?

I don't want this thread to turn into database comparison or a sword fight, below are latest releases from the site you provided :

1-Microsoft Access
Rank # : 10
Latest release : 2019

2- IBM Db2
Rank # : 7
Latest release : 2016

3-Splunk
Rank # : 13
Latest release : None, it was released in 2013 and never been updated

4-Hive
Rank # : 15
Latest release : 2019

5-Amazon DynamoDB
Rank # : 16
Latest release : 2019

6-Teradata
Rank # : 18
Latest release : 2019

7-SAP Adaptive Server
Rank # : 23
Latest release : 2019

And the list goes on, Most of the database engines either paid or open source their latest releases were 2 years ago and most of them never been updated, This information is from the site you provided :

DB-Engines Ranking

Popularity ranking of database management systems.

db-engines.com

Top 2 by what criteria? Price? According to the list i link below, which is sorted by popularity and seem very updated (Wikipedia also references it, so I'm going to assume it's a respected site), we have MySQL on spot #2, well above MS SQL. MySQL was bought by Oracle a couple of years ago, which caused the original developers to fork it into MariaDB, which still is free. And is on place 12 on the list. Just to take an example of a popular database that is free. And let's not forget about PostgreSQL which many consider an absolute piece of art when it comes to databases.

Oracle :

The above costs are per user and there is an extra cost if there are multiple servers, automated backups,managed services, Oracle Forms, etc...

Feel free to read about it from the below link :

https://www.oracle.com/assets/technology-price-list-070617.pdf

 

[sfsameer]

do not blaspheme!

If i combine my team, myself included, another 50,000 developers, 9 companies, a wizard, a god
We wouldn't be even close to Erel

 

[amorosik]

1.000.000 row, in 500.000 byte db
So some form of compression will have to be used
Why?
Why implement data compression, which will inevitably take cpu time, when the size of the storage devices no longer limits the size of the data you want to make permanent?

 

[sfsameer]

which will inevitably take cpu time

We developed our own compression method, been tested and it didn't even take 2% of the CPU and it's been tested on Core i3 which means the CPU was very slow.

When you purchase a VPS, usually they offer 30-40 GB which is almost nothing and it can be full very fast.
Creating very small database files means you have a lot of space to use for other things for example image hosting or a personal website rather than purchasing a VPS solely used for a database engine.

Our second aim is to make it run on free servers as well (Apache), usually free servers offer 500MB disk space which is very small.

 

[Sandman]

8- 10 Millions of tables and rows and the database result will result in less than 500KB file.

That's amazing! You must have come up with those numbers by doing internal testing. Would it be possible for you to provide one of the tables with 10 millions of rows that you can compress to less than 500KB? Perhaps as a CSV? I'd love to see it.

We developed our own compression method, been tested and it didn't even take 2% of the CPU and it's been tested on Core i3 which means the CPU was very slow.

You mention it so casually, but it's very likely that this is an even bigger achievement than your database. You should consider describing it in a document and send to scientific journals, which likely would make you famous. Or at the very least package it as its own product and license it other companies - you should be able to make a lot of money that way.

 

[sfsameer]

You mention it so casually, but it's very likely that this is an even bigger achievement than your database. You should consider describing it in a document and send to scientific journals, which likely would make you famous. Or at the very least package it as its own product and license it other companies - you should be able to make a lot of money that way.

Not really, it's been taught for ages, start from here, study it, understand it and you will manage to create your own method :

Huffman coding - Wikipedia

en.wikipedia.org

 

[LucaMs]

1.000.000 row, in 500.000 byte db

Half byte per row. What is each row made of... air? ?

 

[Sandman]

Not really, it's been taught for ages, start from here, study it, understand it and you will manage to create your own method :

Huffman coding - Wikipedia

en.wikipedia.org

That's funny, because I thought you developed your own method. Instead it seems you implemented something that has been taught for ages.

You forgot to mention the CSV, please remember to share it. Surely you have your testfiles available somewhere nearby. I'm really super interested in seeing how you compress 10 millions of rows into less than 500 000 bytes using your implementation of Huffman coding.

 

[sfsameer]

That's funny, because I thought you developed your own method. Instead it seems you implemented something that has been taught for ages.

Did you read it? literally the first line :

It's a teaching, a theory, an algorithm, it's In 1951

You forgot to mention the CSV, please remember to share it. Surely you have your testfiles available somewhere nearby. I'm really super interested in seeing how you compress 10 millions of rows into less than 500 000 bytes using your implementation of Huffman coding.

It's not a CSV.

Please stop the clowning and the assumptions.

 

[sfsameer]

Half byte per row. What is each row made of... air? ?

Bits.

Once we have concluded the first release, we will release more information on how everything works in the background, why it's very small, why it's very fast, and other information that created a confusion here.