B4J Question How to write a B4x Transpiler?

[Mashiane]

Hi there

You write code in B4x, it comes out as Native Android. It also comes out as Native Objective C or is it Swift? and then... Native Java that works on Linux, MacOS etc. Newly is BANano, which produces Native JavaScript. Awesome stuff.

Ok, I haven't used B4R and not sure what Arduino is, perhaps C++, you get the point.

Point is. I'd like to learn Dart, Python, C#, or whatever tickles my fancy as part of my bucket list. Thing is, I wish I could do it in b4x still.

Its often said that people who don't want to move out of their comfort zone and are stuck with the same stack are in-experienced. Well ok. In-experienced in Dart, Python, C# or whatever, granted and Im speaking for myself anyway. Still anyway, I'd like to still find myself doing whatever programming language I wish using B4x.

So how can I go about writing a b4x transpiler? Pitfalls, Pros and Cons, best practise?

In my native African language Xhosa, we say, "Inyathi ibuzwa kwabaphambili" meaning "the insights about the journey is ought to be sought from those who have travelled it"

Thank you in advance.

 

[Sandman]

Some very naïve thoughts on the topic.

There are (at least) two different ways to make a transpiler for this. Either you do it inside the IDE, just like what Erel provide today, or you do it from the outside, which is what I imagine @alwaysbusy is doing for his web solutions.

For the latter, I imagine @alwaysbusy would be the expert on how to do that, so he should be able to answer all your questions. (If he has resources to do so, obviously.)

For the first, I know I have seen Erel say that the IDEs share the majority of the code between them. My mental model for that is that he actually have three parts for each IDE:

• IDE code - identical code for all platforms
• UI code - one for each platform - to cater for the differences between the IDEs in the user interface
• Transpiler code - one for each platform
  • Contains syntax definitions and whatnot to output to Java, ObjC etc

Assuming this is somewhat correct, it would be technically possible for Erel to make the transpiler code into a .dll and place it into the IDE folder. And if he also published some documentation for it, it would be possible for the community to then make their own dll, replacing the standard one, and allow for transpiling to Dart, Python, C#, or whatever tickles the developers' fancy.

I can't see that ever happening though, nor would it be a very good idea to do so. I'm just saying I think it would be technically possible.

Bottom line, if you want to make transpiling happen, do it outside the IDE and ask @alwaysbusy for advice.


PS. According to Google Translate, the phrase "Inyathi ibuzwa kwabaphambili" means "The buffalo is questioned in the forefront". I'm obviously not saying your translation is wrong, I just found it amusing that Google translated it without obvious problems, to something that in some way kind of means a bit what you wrote. Plus they made it sound funny.

 

[Mashiane]

"Inyathi ibuzwa kwabaphambili".

I'm am very sure that googles AI does not understand Xhosa idioms.

I hope both can pitch in on this. At least we are not asking for them to share "trade secrets" (I hope), but just methodologies / best practise etc.

Without dlls would be the preferred option for me, preferably within the existing b4x b4j environment, hopefully that's possible.

 

[Mashiane]

Please lets not pollute this thread with irrelevant language intricacies, this thread is not about idioms but about an important question.

 

[agraham]

Point is. I'd like to learn Dart, Python, C#, or whatever tickles my fancy as part of my bucket list. Thing is, I wish I could do it in b4x still.

I don't understand what you mean here. If you want to program in C# then you will be writing C# and B4X becomes irrelevant. If you write in B4X and transpile to C# then you are not really learning to program in C#.

EDIT: Or do you mean that you want to write a transpiler in B4X to generate C# code?

 

[Mashiane]

EDIT: Or do you mean that you want to write a transpiler in B4X to generate C# code?

Absolutely, actually the idea is just learn about how one can do that in B4x, whether for c#, dart, python etc. Challenges, how they were overcome, stuff like that.

 

[aeric]

Learning on how to transpile one code to another code will be useful for example I want to create a new web framework like BANano but not dependent on BANano as the core library.
I think the idea is similar to what I have done in my Web API Template to generate the HTML and JavaScript Help file. Maybe I am wrong. I will read the source code from language A (line by line) and capture the reserved keywords and matching certain patterns then doing some "magic" and finally output something else in language B.

 

[alwaysbusy]

A transpiler like BANano converts ALL the logic code written in B4J (supports about 99% of B4Js keywords) to JavaScript. Unlike @aeric's Web API Template (fantastic tool btw!) which only transpiles the method definitions and none of the logic inside the methods.

BANano first rewrites the B4J code to a uniform format (not everyone has the same 'style' of programming, and B4J allows quite some freedom ).
Then the code is cleaned up through several phases to make everything ready to be parsed. Next every line of code is converted to symbols (like the words in a sentence) where each word is identified so the Transpiler knows exactly what kind of 'word' it is working with, especially variables. From that, BANano builds some kind of 'logic' tree of the complete project (so it knows which methods are called by who among other things). Next this tree is optimized, transformed and prepared for the target language. Now BANano is ready to walk this tree and generate the corresponding JavaScript code. Finally, this 'translated' tree is written to runnable JavaScript code.

Through the years, new features were introduced to BANano like the support of the B4J Abstract Designer, Debugging features and the possibility to use the .b4xlib BANanoLibraries.

A very good knowledge of the source and target language is absolutely crucial: so writing a Transpiler to learn another language is not a good idea. Before I wrote BANano, I had written quite some WebApps manually in JavaScript and had written several file format converters so I was well prepared of what was coming. Even then, there were quite some bridges to cross in parts where the B4J syntax and the JavaScript syntax where nothing alike. B4J has some cool language features that no other language has, but they can mean trouble to transpile ? ! Not knowing exactly how the target language works will definitely result in an unstable exercise. The interweb is full of such failed attempts. It really isn't hard to get 'something' working, it is very hard to get it 'production-usable' working.

Writing a Transpiler is very close to writing a Compiler: one generates another source code, the other writes bytecode but the process is very similar.

So my advice would be: if you want to write a B4X2Dart Transpiler for example, LEARN Dart FIRST, make MANY programs with it. Then you may be ready to write the Transpiler. The idea to write a 'BANano 4 Python' has been in my mind for quite a while, and although I have written several Python apps for my day job over the last few years, I'm 100% sure I am not ready to take on that endeavor yet.

Alwaysbusy

 

[imbault]

I wrote a C++ preprocessor and participate in creating a compiler many years ago (Nantucket Clipper), this is very interesting

Good reading :
https://tomassetti.me/how-to-write-a-transpiler/

Patrick

 

[aeric]

I wrote a C++ preprocessor and participate in creating a compiler many years ago (Nantucket Clipper), this is very interesting

Good reading :
https://tomassetti.me/how-to-write-a-transpiler/

Patrick

Nice info.
To sum up, first we need a B4X parser to parse to abstract syntax tree (AST), one or more transformation work to the target AST and finally use a code generator to spit the code of target language.

From the article:

So our transpiler will have these stages:

1. Parsing stage: we will adopt a parser to obtain an AST from the code of the original language
2. Transformation stage: we will transform in one or more steps the AST of the original language into the corresponding AST of the target language
3. Generation stage: once we have the AST of the target language we generate the corresponding code out of it

 

[Mashiane]

Good reading :

Awesome, thanks for sharing this, its very very informative.