B4J Library jAWTRobot - invoke keyboard and mouse events, etc...

lemonisdead · Feb 18, 2016

Cableguy said:
where can I find the temp batch file

For my part it is created in the DirApp folder. For example when I compile and execute from B4J, it is created in the Objects folder. But you will have to kill the restarting windows (with the timeout) or it will be deleted

Cableguy · Feb 18, 2016

Thanks for the suggestion about the killing, I was able to check the batch file...
Even though the path was correct, apparently it didn't like it much, so after I removed the File.DirApp and the leading slash, it works... so now I can start working on the creation of my own Batch file

Roycefer · Feb 18, 2016

You can also use Robot.relaunchSelfBatch() to relaunch your program via a custom batch file that you write. This is useful for when you have to launch your program with VM args or other special features and tasks beyond just "javaw -jar myjar.jar".

Cableguy · Feb 18, 2016

Thanks Roycefer...

I managed to get it working, and created my own simple batch

ThRuST · Dec 1, 2016

Can I use a hotkey with this lib?

ThRuST · Jan 8, 2017

I have AWTRobot v1.45 (the latest I found) but can not use RobotPaste2, RobotPaste3 etc
Is there a newer version or how else did you get RobotPaste2? RobotPaste is not working on Mac so I ask for help. Thanks

rboeck · Jan 8, 2017

Hi, as i remember, RobotPaste2 and RobotPaste3 are older versions, before in version 1.45 the robotPaste routine was able to work for windows and mac; i made test an both systems in this time. Currently i have no mac to try, but i remember, that ver. 1.45 worked with mac without problems. (for my type of usecase)

ThRuST · Jan 8, 2017

This is how I solved it (jAWTRobot v1.45)

B4X:

Dim os As String = GetSystemProperty("os.name", "").ToLowerCase

If os.Contains("win") Then

        ' Windows
           fx.Clipboard.SetString ("hello")

        AWTRobot.RobotSpecialKeyCombo("ctrl_v")
   
   
    Else If os.Contains("mac") Then
   
        ' Macintosh
        fx.Clipboard.SetString ("world")

        AWTRobot.RobotSpecialKeyCombo("meta_v")

    Else
   
           ' Linux
   
    End If

Dunno about Linux, Solaris other O/S etc every comment is useful thanks

rboeck · Jan 8, 2017

I think, you dont need to check for os; try it this way:

B4X:

If fx.Clipboard.HasString Then
      AWTRobot.RobotPaste(Text)
End If

ThRuST · Jan 8, 2017

As far as I remember that did work well on PC, but not on my MacBook. In the intelisense it also says something like under the RobotPaste

If it does not work use the meta_v on Mac

I tested my solution an hour ago and it worked on both platforms. How about Linux etc?

rboeck · Jan 8, 2017

I have a running Ubuntu, but i am not able to make javafx working...

ThRuST · Jan 8, 2017

found this

The JavaFX SDK runs on Windows and Mac OS X. A beta release of the JavaFX SDK is provided for Ubuntu Linux and the OpenSolaris operating systems.

Go Here

Roycefer · Jan 8, 2017

The jAWTRobot does not rely on JavaFX. You can use it in non-UI B4J apps. For example, the fx.Clipboard.HasString() call is duplicated in the AWTRobot class with robot.ClipboardStringHas().

Users have reported success on Mac OS with the latest version of the jAWTRobot library (1.45) in using the robot.RobotPaste function. If it's not working for you, post the Exceptions you're getting and I'll see what I can do. I don't have a Mac so I can't directly test it.

ThRuST · Jan 9, 2017

Sorry I don't remember if it worked or not since I was solving a problem with the contextmenu staying under applications. However I solved it the way I described so that is an alternative solution I will use in my project, it works great because of your amazing library. In time you will see the result once I publish it.
I just wonder how to detect which mouse button that was clicked. How to do that? inside the mouseclick event?

ThRuST · Jan 9, 2017

Since my b4j project will have mouse button support I am working on a solution to add support for the Logitech G300S gaming mouse which I've bought, so I can hardcode support for 5 buttons (out of 9), the rest will have to use keyboard assignments which is done in the tray application provided by Logitech. Which means I will have to include support for some special keys that can be assigned to the mouse in Logitechs application. Buy the mouse and check my post Here

Roycefer · Jan 9, 2017

Yes, the mouse button info is provided in the NativeHook's _NativeMouseClicked() event. 1 is the left button, 2 is the right button and 3 is the center button. I don't know if there's support for numbered mouse buttons beyond that; my mouse only has 3 buttons. That's probably an OS-level question; I suspect the library will support whatever the OS supports.

If your Logitech mouse supports mapping some buttons to key combos (like ctrl_alt_f, though you can also listen for one-key "combos"), you should check out my KeyComboListener class. It greatly simplifies using NativeHook to listen for key combo events. It also allows for easy run-time creation of new key combos.

ThRuST · Jan 9, 2017

I have allready tested your hook library with my mouse and it's just to use numbers 1-5 for default buttons, which are 1=left, 2=right, 3=middle, 4 and 5=Buttonx1, buttonx2 (Microsoft also provide support for this in visual studio). I think that's why Logitech provides the binding application to use shortcuts to bind the rest of the buttons. Recently I got the idea that programmers can use a gaming mouse and benefit from using the many mouse buttons, and that goes well along with my new project which is a knowledge management system, so contextmenus will play a vital keyrole in how users will interact with development environments.
It's a bold chance by me to challenge traditional methods, but after watching numerous of interviews with Steve Jobs I understand that innovative ideas combined with the latest technology helps to push the industry forward. He also said that the hard working individuals are aware of this and intentionaly seek others who wants to do the same thing, which is just what is happening here

The difficult part is to convince people who are used to 'the old ways' that technology is of great help and again there will be the people who strive for the better solution who is first to use it. Programming is one of the most powerful knowledges one can use in todays modern civilazations so it's a great thing that we evaluate the mouse, which will probablt be exchanged to eye tracking once the prices goes down, but then again currently there's sceptical people to that technology as well, which only makes us have to wait until they know they won't get eye cancer from it!! However with my mouse idea, programmers will have to get used to the many buttons, but the most you get used to it the more efficient the system will assist you. Since I am evaluating it by hard testing it with CSS driven contextmenus, I have discovered that the more complex designs I use in CSS, the slower the refresh is on the screen. I am not saying your code is to blame but since you probably had no idea about that I was going to do this, I recommend you to play around with my second design example and add more complexity to it, and then you click on it. Perhaps we have to tell Steve05 this since he provided the FloatingContextmenuv06' example and I made some CSS templates it's only good if it can be evaluated and perhaps updated. I wanted to tell you this since you created the hook, which is excellent as it is but maybe it can be tweaked a bit, so please take a look at it and let me know what you think.

EDIT: As the menus might get larger than in the example code either Steves code can be optimized or/and the hook library as well. Thanks for looking into this.

Roycefer · Jan 9, 2017

It's almost certainly not the NativeHook code that is slowing down. If you're calling code that modifies the UI based on NativeHook mouse clicked events, then the UI-related code isn't even running in the same thread as the NativeHook code.

I recommend you do some fine-grained benchmarking in your code to see where the slowdown is occurring. Example:

B4X:

Dim d as Long = DateTime.Now
'Execute some code that we'll call "Step 1"
Log($"Step 1 took ${DateTime.Now-d} milliseconds"$)
d = DateTime.Now
'Execute some other code that we'll call "Step 2"
Log($"Step 2 took ${DateTime.Now-d} milliseconds"$)
'so forth and so on....

You can see how the duration of each Step changes as you vary some other variable (CSS complexity, perhaps).

If millisecond precision is insufficient, use robot.SystemNanoTime() for nanosecond precision:

B4X:

Dim robot as AWTRobot
'....
Dim d as Long = robot.SystemNanoTime
'Execute some code that we'll call "Step 1"
Log($"Step 1 took ${robot.SystemNanoTime-d} nanoseconds"$)
'so forth and so on....

ThRuST · Jan 9, 2017

Wow, so you have allready baked that in as well, amazing

Now I want a bench marking application to check my system. When I posted my answer I was thinking about that I use a rather slow computer!!! it's a Packard-Bell EasyNote TE 1.7 celeron!!!!!

so it could be that, or what do you think haha
Since you have it in the library do you mind putting together a little form with a benchmark value and post here. Then everybody can use it. Great idea!!
Not strange my CSS styled contextmenu took resources since there were 3 contextmenu clones at the same time when clicking fast!!!

Most users don't use a 1.7Ghz computer anymore, and it's just a matter of time until they end up on the garbage mountain (like my code) just kidding a bit.
Your code does not need to be changed now we know celeron computers is not good for complex CSS themes.

Btw another question I still want to know, how can I catch which mouse was clicked in the MouseClick event? I tried if button = 2 then... but that didn't work.
I guess I made a simple misstake.

Roycefer · Jan 9, 2017

Which button was clicked is a field of the NativeMouseEvent Object passed as an argument in the _NativeMouseClicked() event:

B4X:

Sub nh_NativeMouseClicked(nme as NativeMouseEvent) As Boolean
   If nme.MouseButton==2 Then
         Log("Right mouse button was pressed.")
   End If
   Return False
End Sub

All the relevant info about the event will be stored in fields of nme. Use the code completion in the IDE to see what's available.

As for benchmarking, the examples I posted aren't for benchmarking a computer but for benchmarking specific sections of code.

B4J Library jAWTRobot - invoke keyboard and mouse events, etc...

Attachments

Well-Known Member

Expert

Well-Known Member

Expert

Well-Known Member

Well-Known Member

Well-Known Member

Well-Known Member

Well-Known Member

Well-Known Member

Well-Known Member

Well-Known Member

Well-Known Member

Well-Known Member

Well-Known Member

Well-Known Member

Well-Known Member

Well-Known Member

Well-Known Member

Well-Known Member

Similar Threads