01 / Human–robot interaction
SHRIMP
Iterative Refinement of Robot Task Plans
Research assistant · system & interaction design
Can people correct a robot’s interpretation before it becomes physical action?
A robot can produce an executable plan that still misunderstands a person’s intention. SHRIMP lets non-expert users inspect, revise, and simulate tabletop task plans before authorizing the physical robot.
My contribution connected perception and robot–interface integration with interaction design, testing, study execution, and collaborative writing. I worked within the team building and evaluating the complete system.
Study apparatus / physical team system
01 / My contribution
Connecting the physical system to human inspection.
Perception + physical system
I worked on RGB-D perception, camera calibration, object localization, and point-cloud integration for real-robot execution. These responsibilities connected the physical scene to the system people inspected.
Interaction + integration
I contributed robot–interface development and editable-plan interaction design, then integrated and tested the connected system. The work spanned the visible interface and the robot behavior behind it.
Research + evaluation
I helped set up and run study sessions, collected data, and contributed to manuscript writing and revision. My role included the practical work of making the system available for human-subjects research.
The integrated system and research results belong to the team. I do not claim sole architecture, study leadership, or individual analysis ownership.
02 / Problem & process
Make the interpretation available for revision.
Workflow summary / team system
- 01Ground the scene
Locate objects.
- 02Generate a plan
Expose action primitives.
- 03Preview
Test in simulation.
- 04Inspect & revise
Edit, then preview again.
- 05Execute
The person authorizes action.
The team’s interaction supports both language re-prompting and explicit changes to primitive order and parameters. Instead of asking people to describe every correction in words, it makes the generated action structure available for direct inspection.
The parameter editor is one concrete example: arm, object, grasp pose, and end position are exposed together. Simulation gives a way to examine the revised manipulation before committing to the physical scene.
Inspect the full interface
03 / Outcome & reflection
More agency also has an interaction cost.
35 participants / within-participant comparison
The team compared Plan+Edit, Plan-Only, and No-Plan on tabletop kitchen tasks. Plan+Edit produced significantly higher perceived control and transparency than No-Plan, but higher workload than Plan-Only. Other pairwise control/transparency comparisons were not significant; neither were usability or task-success differences across conditions.
For me, the finding sharpens the design problem: making behavior editable offers agency, but also asks the person to do more work. A useful interface must account for the cost of intervention alongside the opportunity to intervene.
My next design question would be how to keep detailed correction available while reducing unnecessary inspection for simpler tasks. That is a direction to evaluate, not a demonstrated improvement.