Air mouse App

[walterf25]

Hello everyone, i'm working on an application which consists of a server on my laptop and a client on my android phone, i'm using the UDP library to send and receive data packets, so far everything works fine, i can turn up/down the volume on my laptop through my phone, i can also move the mouse around through touching the screen on the phone. This is where i need some help, when i slide my finger on the touchscreen on the phone, i send those x and y values to the computer therefore moving the mouse on the laptop. My problem is that let's say i move the mouse from left to right, since the screen resolution is smaller on the phone than on the computer, the mouse will obviously only move to the right as far as the width of the phone's screen, i need to figure out a way to let's say if i lift the finger and go back to the left and slide the finger from left to the right again, to keep the mouse moving to the right from where it left off when i lifted my finger up.
I kind of made my self dizzy explaining this, hope i make any sense, i hope i can get some help with this.

thanks everyone,
Walter Flores

 

[admac231]

I would say there is 2 ways of doing this.

1. 1. Get the resolution of the phone and the computer. Set a scale. For example, 480*800 on phone, 1920*1080 on computer. So when x on phone screen = x*4 on computer screen and y on phone screen is equal to y*1.35
2. 2. Instead of setting mouse x,y to screen touch x,y get the change between the original x,y and new x,y. SO when the user swipes from - say - 50,50 to 150,150 you move the mouse 100x and 100 y

I hope this makes sense.

 

[walterf25]

Hi Admac213, thanks for the reply, actually i don't really understand the second option, do you think you can elaborate a little more about it, I already tried the first option where i figure out the resolution on the computer and on the phone and must multiply the x and y coordinates by that scale, but i have seen other apps where they you can just keep going from where you left off.

thanks, any help will be greatly appreciated.

 

[admac231]

Well, when you capture a touch event (on a panel I'm assuming...?) you store the initial x,y co-ordinates. Then as the user moves there finger you work out the difference of the new x,y co-ordinates and move the mouse accordingly.

For example:

Dim origX As Float
Dim origY As Float
'Possibly set these to initial mouse x,y co-ordinates on app load?

Sub Panel1_Touch (Action As Int, X As Float, Y As Float)
   If x <> origX Or y <> origY Then
      If x < origX Then
         'Move mouse to mouseX-(origX-x)
      Else If x > origX Then
         'Move mouse to mouseX+(x-origX)
      End If
      
      If y > origY Then
         'Move mouse to mouseY-(origY-y)
      Else If y < origY Then
         'Move mouse to mouseY+(y-origY)
      End If
   End If
   origX = x
   origY = y
End Sub

This was written on the forum so apologies if the syntax is off.

 

[walterf25]

Ok i got it, but what is mousex and mousey values?

 

[admac231]

That would be the current x,y of the cursor on the computer screen. You would have to request it from your server app somehow.

 

[walterf25]

Hi there, thanks for your help, i've been playing around with your code but it doesn't give me the results i want, i'm getting the current position of the cursor on the computer and using it like you suggested in your code.

this the code where the packets are received and then i split the x and y coordinates.

Sub UDP_PacketArrived (Packet As UDPPacket)
  Dim msg As String
  msg = BytesToString(Packet.Data, Packet.Offset, Packet.Length, "UTF8")

label1.Text = msg
headerlen = msg.Length 
header = Regex.Split("~", msg)

X1 = header(0)
Y1= header(1)
End Sub

i then assign this values to Mousex and Mousey accordingly, i can see the cursor on the computer move but it moves only a little bit.

what do you think i'm doing wrong?

thanks,
Walter

 

[JonPM]

Have you tried converting the x y coordinates to a percentage then translating them onto the computer?

Sent from my DROIDX using Tapatalk 2

 

[walterf25]

Yes i've tried that, and it works fine, the issue is this....
let's say you, as the user want to scroll the mouse from left to right, the mouse will move a certain distance right, but let's say that your finger is all the way to the right on the phone's screen, if you want to keep moving the mouse to the right then you would have to lift your finger up and go back to the leftmost side of the screen on the phone to keep moving the mouse farther, the issue is this that when you do this the mouse also goes back to the previous position starting from the original position, i tried this and it seems to work, is not that smooth but it works ok.

first i ask for the computer's screen resolution

Sub UDP_PacketArrived (Packet As UDPPacket)
  Dim msg As String
  msg = BytesToString(Packet.Data, Packet.Offset, Packet.Length, "UTF8")


headerlen = msg.Length 
header = Regex.Split("~", msg)
'
'x2 = header(0)
'y2= header(1)

screenheight = header(0)
screenwidth = header(1)

phoney = lv.Height
phonex = lv.Width

scaley = screenheight / phoney
scalex = screenwidth / phonex


label1.Text = "H: " & screenheight & " W: " & screenwidth
End Sub

i figure out the scale and then when i on the Activity_Touch sub i write the coordinates back to the computer multiplying the coordinates times the scalex and scaley, but i still have the original isssue.

 

[JonPM]

I mean a more direct method.
For example:
Phone dimensions:
W = 480
H = 800

Computer dimensions:
W = 1280
H = 800

User taps point on phone:
x = 380
y = 750

Calculate ratio:
x = 380/480 = 0.7916
y = 750/800 = 0.9375

Transmit ratio to computer and multiply by computer's res:
x = 0.7916*800 = 633
y = 0.9375*1280 = 1200
new x/y coordinates 633, 1200

 

[walterf25]

That is exactly what im doing and it works fine but that doesn't take care if the original issues

 

[JonPM]

Hmm, maybe I'm confused then. I don't understand why the mouse cursor won't move to the edge of the screen when your finger is at the edge of the phone. If you are touching the right most edge (i.e. 480) then it should move the mouse cursor to the right edge also since the ratio is 1 (and on the computer should register as 1280).

Sent from my DROIDX using Tapatalk 2

 

[walterf25]

uhmmm, i think i'm not explaining myself correctly,

what i want to accomplish is this......
let's say that i move the mouse only to the center of the screen rather than all the way to the rightmost part of the screen.
imagine the mouse/cursor in the center of the screen, now i want to move the mouse farther more to the right, when i go back to the phone's screen and start dragging my finger across the screen the mouse/cursor will start from the beginning again since i started from the leftmost side of the phone's screen again.

ugh, i don't think i know how to explain this!!!!

sorry!

 

[JonPM]

I see. What you need to do then is send the computer only the DIFFERENCE of pixels dragged on the phone instead of the actual x y coordinates. Also on the touch released code you should set the mouse's x y coordinates as 0,0 regardless of where it is on the screen.

Sent from my DROIDX using Tapatalk 2

 

[walterf25]

Oh wow, i just got it working just the way i want, thanks for the help jonPM and admac231, here's how i got it to work,i kind of modified a little adma231 code, and it works perfect
again thanks everyone for your help.

Sub Panel2_Touch (Action As Int, X As Float, Y As Float) As Boolean 'Return True to consume the event
      Dim Packet4 As UDPPacket
   Dim Data4() As Byte
   
Log("Action " & Action)
       Select Action
        Case Activity.ACTION_MOVE
      X3 = X - Origx + X3    'get the difference from the origx and x and increment by that value
      y3 = Y - Origy + y3    'get the difference from the origy and y and increment by that value
      X1 = X3               'assign value of x3 to string x1
      Y1 = y3               'assign value of y3 to string y1
      If X3 < 0 Then         'set lower boundary for x coordinate
      X3 = 0
      End If
      If X3 > screenwidth * 2 Then   'set higher boundary for x coordinate, i'm using two monitors so i set the high boundary to 1024 * 2
      X3 = screenwidth * 2
      End If
      If y3 < 0 Then            'set lower boundary for y coordinate
      y3 = 0
      End If
      If y3 > screenheight  Then      'set higher boundary for y coordinate
      y3 = screenheight 
      End If
             coordinates = "xpos"&X1&"ypos"&Y1   'assign x, and y coordinates to coordinates string
         Data4 = coordinates.getbytes("ASCII")
         Packet4.Initialize(Data4, "10.3.57.20", 11000)
         UDPSocket.Send(Packet4)           'send data
       ' Case Activity.ACTION_UP
      'x2 = X
      'y2 = Y
      
      Case Activity.ACTION_DOWN
      Origx = 0   'on first tap set x coordinates to 0
      Origy = 0   'on first tap set y coordinates to 0
      End Select
      Origx = X   'get initial x location
      Origy = Y   'get initial y location
Return True
End Sub

hope this helps someone in the future.

cheers,
Walter