Rob's Tech Thoughts

Arduino Due Virtual Keyboard Upgrade: Enhanced Control and Interruptibility

Posted by:

Rob

|

On:

|

In my previous blog post,, I introduced My project where an Arduino Due acted as a virtual keyboard—I called the “RoboKeyboard”—controlled by a Kotlin PC program. This setup allowed for keyboard macros and testing by sending commands from the PC to the Arduino, which would then emulate keystrokes to a connected device.

While the initial version provided a solid foundation, it lacked the ability to handle interruptions during lengthy operations. This limitation could lead to synchronization issues between the PC program and the Arduino, especially when sending large amounts of data or complex commands.

In this blog post, I’ll walk you through the enhancements made to the original code, focusing on:

  • Introducing interruptible commands
  • Implementing a busy signal mechanism
  • Updating command parsing and execution for better responsiveness

Overview of the Original Code

The initial version of the Arduino sketch provided basic functionalities:

  • Command Parsing: The Arduino read commands from the serial port and executed them accordingly.
  • Keyboard Emulation: It could send keystrokes, key combinations, and perform edit actions like cut, copy, and paste.
  • Status Reporting: The Arduino could respond to basic status inquiries like PING and STATUS.
  • Configuration Options: It allowed settings for delays, key press lengths, and enabled or disabled features like echo and debug messages.

However, the code executed commands in a blocking manner. Once a command started, it had to complete before the Arduino could process any new input. This behavior was not ideal for a responsive system, especially when dealing with long text inputs or complex operations.

Reason for choice of Arduino Due

The Arduino Due is quite an old board now but still available and useful for this project as it has two USB ports one of which can emulate a keyboard. I chose it as I happened to have one spare from a few years back. It would be possible to replace this board with two different boards connected together communicating via serial links. If it needed to be used with a different board however. It would be easier to use a more modern board with the same 2 USB ports one of which can emulate a keyboard.

New Features and Improvements

1. Interruptible Commands

The updated code introduces the ability to interrupt ongoing commands. This means that if the Arduino is in the middle of processing a long command, such as typing a lengthy text, it can now stop immediately upon receiving an interrupt signal.

You can now interrupt ongoing commands using the CMD:STOP and CMD:PAUSE commands. CMD:STOP halts the current action and clears any pending text, while CMD:PAUSE temporarily suspends typing without losing data. This allows for more control over RoboKeyboard’s actions.

2. Non-Blocking Operations

Long actions like typing large amounts of text are now handled in a non-blocking manner. This prevents RoboKeyboard from becoming unresponsive during these operations, allowing it to react to other commands more efficiently.

3. Busy Signal Mechanism

RoboKeyboard now sends a “busy” signal to the controlling Kotlin program when it’s processing a command. This prevents the program from sending additional commands while RoboKeyboard is occupied, ensuring smoother operation and preventing input conflicts.Long actions like typing large amounts of text are now handled in a non-blocking manner. This prevents RoboKeyboard from becoming unresponsive during these operations, allowing it to react to other commands more efficiently.

4. Improved Command Parsing and Execution

The command parsing logic has been refactored for better efficiency and scalability.

Key Changes:

  • Normalized Commands: Commands are normalized to a standard format, replacing delimiters like _, -, ., ,, and spaces with colons (:). This unification simplifies parsing.
  • Command Lookup Table: A structured array (knownCommands[]) holds all valid commands with their associated types and parameter expectations.
  • Non-Blocking Actions: Actions are now represented by a state machine (Action struct and ActionType enum), allowing the main loop to process them incrementally without blocking.

5. Improved Status Reporting

The STATUS command now provides a more detailed report, including the device’s state (connected/disconnected, ready/busy, paused), debug and echo settings, and version information. This enhanced feedback gives you a clearer picture of the Arduinio Due’s status.

Detailed Explanation of Changes

Command Struct and Normalization






struct Command {
  const char* name;
  CommandType type;
  bool hasParameter;
};





    

  • Purpose: Simplifies command parsing by associating command strings with their types and whether they expect parameters.
    • 




  • Normalization Function: Converts incoming commands to uppercase and replaces various delimiters with colons.
    • 






Interruptible Actions





Action State Machine





enum ActionType {
  NONE,
  SEND_MESSAGE,
  TYPE_LOREM_IPSUM,
  // ... other action types
};

struct Action {
  ActionType type;
  String message;
  int index;
  // ... other fields
};





    

  • Non-Blocking Execution: By representing actions as states, the main loop can process parts of actions incrementally.
    • 




  • Interrupt Checking: Before each incremental action, the code checks if an interrupt signal (isStopRequested flag) has been set.
    • 






Stop Command Implementation










case CMD_STOP:
  isStopRequested = true;
  currentAction.type = NONE;
  isBusy = false;
  Keyboard.releaseAll();
  sendStatusResponse();
  break;





    

  • Functionality: Immediately stops the current action, resets the action state, and releases any pressed keys.
    • 




  • User Feedback: Sends a status response to inform the PC program of the new state.
    • 






Busy Signal Reporting





Status Response Enhancement










void sendStatusResponse(bool fullStatus = false) {
  Serial.print(F("RESPONSE:{\"app\": \"RoboKeysDuino\", \"paused\": \""));
  Serial.print(isPaused ? "yes" : "no");
  Serial.print(F("\", \"busy\": \""));
  Serial.print(isBusy ? "yes" : "no");
  // ... additional status fields
  Serial.println(F("\"}"));
}





    

  • Fields Added: paused and busy fields provide real-time status.
    • 




  • Integration with PC Program: The Kotlin application can parse this JSON-formatted string to adjust its behavior accordingly.
    • 






Improved Command Parsing Logic





Normalization Function










String normalizeCommand(const String& command) {
  // ... code to replace delimiters and convert to uppercase
  return String(buffer);
}





    

  • Purpose: Ensures that commands are in a consistent format for easier matching.
    • 




  • Benefit: Reduces parsing errors and allows for more flexible input from the user.
    • 






Command Lookup










Command* findCommand(const String& command, String& parameter) {
  String normalizedCommand = normalizeCommand(command);
  // ... code to find and return the matching command
}





    

  • Efficiency: By iterating over the knownCommands[] array, the code can quickly identify valid commands and extract parameters.
    • 






Non-Blocking Main Loop










void loop() {
  // ... code to read serial input
  processCurrentAction();
}





    

  • Separation of Concerns: The processCurrentAction() function handles ongoing actions, while the main loop remains responsive to new input.
    • 




  • Interruption Checks: Before executing parts of an action, the code checks if it should stop or pause.
    • 






Action Processing Function










void processCurrentAction() {
  if (currentAction.type == NONE || isStopRequested || isPaused) {
    // ... code to handle idle or interrupted states
    return;
  }
  
  switch (currentAction.type) {
    case SEND_MESSAGE:
      // ... incremental processing of the message
      break;
    // ... other cases
  }
  
  // ... code to update busy status
}





    

  • Incremental Execution: Actions like sending a message are processed one character at a time, allowing for interruption.
    • 




  • State Updates: The busy status is updated based on whether an action is in progress.
    • 






Additional Commands





Pause and Resume










case CMD_PAUSE:
  isPaused = true;
  isBusy = false;
  sendStatusResponse();
  break;

case CMD_RESUME:
  isPaused = false;
  isStopRequested = false;
  break;





    

  • Pause (CMD:PAUSE): Temporarily halts ongoing actions without losing the current state.
    • 




  • Resume (CMD:RESUME): Continues processing from where it was paused.
    • 






How the Improvements Enhance Functionality





Benefits of Interruptible Commands





    

  • Responsiveness: The Arduino can now react immediately to critical commands, improving user experience.
    • 




  • Error Handling: If the PC program detects an issue, it can send a stop command to prevent unwanted behavior.
    • 




  • Flexibility: Users can adjust settings or change commands on the fly without waiting for long operations to complete.
    • 






Ensuring Smooth Communication





    

  • Busy Signals: By informing the PC program of its current status, the Arduino helps prevent command collisions and data loss.
    • 




  • State Synchronization: The enhanced status reporting ensures that both the Arduino and PC program have a consistent understanding of the system state.
    • 






Conclusion





The updated Arduino code for the RoboKeyboard significantly improves its robustness and usability. By introducing interruptible commands and a busy signal mechanism, we ensure smoother operation and better coordination with the controlling Kotlin program.





Key Takeaways:





    

  • Non-Blocking Operations: Transitioning to a state machine approach allows for responsive and interruptible command execution.
    • 




  • Enhanced Communication: Busy signals and detailed status reports keep the PC program and Arduino in sync.
    • 




  • Scalable Design: The refactored command parsing logic makes it easier to add new commands and features in the future.
    • 






Future Improvements:





    

  • Error Recovery: Implementing retries or error correction mechanisms.
    • 




  • Advanced Commands: Adding support for macros or complex command sequences.
    • 




  • Security: Introducing authentication for commands to prevent unauthorized access.
    • 






By addressing the limitations of the original code, we have created a more robust and user-friendly virtual keyboard system, paving the way for further enhancements and applications.















Downloadable Code





You can download a Kotlin file containing all the extension functions mentioned in this post here.





You will be able find the complete code and documentation on my GitHub repository. I welcome feedback, contributions, and suggestions for improvement.





License





This code is licensed under the GNU General Public License v3.0 or later.
You can find a copy of the license at https://www.gnu.org/licenses/gpl-3.0.en.html








	
	
	
	
	
	
	

	
Posted by

	
	
	
Rob

	
	
	
in