This sounds sort of like what one of my apps does.
I have an app that is designed to link to an eStim unit, which has a serial port. The connection from the Android device is via a cable with the FTDI chip in it (and a USB host adaptor). To allow remote access, the app also listens on a TCP port for incoming connections, and when one is received, it passes all data from the TCP port to the serial, and vice versa.
Does that sound fairly similar to what you're doing?
Obviously, there's a lot of extra bits going on in my code (not least because the app can be used to connect to as a client via TCP, as well as a server), but the fundamentals aren't too tricky.
I use AsyncStreamsText to process the data, and the USBSerial library to drive the cable.
I have two services in the app - one is called 'estim', and handles the communication between device and the estim unit over the local cable. This is how it works (and sets up the server option, which is via the 'listener' service
	
	
	
	
	
	
	
	
	
		Dim stimStream As AsyncStreamsText
Dim connected As Boolean = False
 
Dim estimUSB As UsbSerial
	 
	
	
		
	
 
	
	
	
	
	
	
	
	
	
		        If estimUSB.open(9600) = estimUSB.USB_DEVICE Then
           stimStream.Initialize(Me,"stimStream",estimUSB.GetInputStream,estimUSB.GetOutputStream)
           connected = True
          
           ' start the listener, if necessary
           If StateManager.GetSetting2("enableServer","") = "yes" Then
             ' start the server if necessary
             Select StateManager.GetSetting("listenVia")
               Case "BROKER"
                 startBroker
               Case "PEBBLE"
                 If pk.isWatchConnected = True Then
                   pk.startAppOnPebble(APP_UID)
                 End If
               Case Else
                 StartService(listener)
             End Select
           End If          
         Else
           connected = False
           ToastMessageShow("Error opening USB connection",True)
         End If
	 
	
	
		
	
 
In the service start sub for the listener, I initialise the connection like this:
	
	
	
	
	
	
	
	
	
		    serverTCP.Initialize(portNumber,"tcpServer")
     serverTCP.Listen
     serverIP = serverTCP.GetMyWifiIP
     ToastMessageShow("Listening for connections on " & serverIP,True)
	 
	
	
		
	
 
And in the NewText sub for the tcp stream, I pass the received data back to the estim service, where a function called sendStimCommand passes it to the USB stream with the .write function for the AsyncStreamText object
	
	
	
	
	
	
	
	
	
		CallSub2(estim,"sendStimCommand",Text)
	 
	
	
		
	
 
Similarly, in the NewText sub for the serial stream, I call a sub in the listener to push the data back through the TCP port:
	
	
	
	
	
	
	
	
	
		  If listener.serverConnected Then
     CallSub2(listener,"sendResponse",stimData)
  End if