Robotic Arm Work Cell with Force-Sensing Closed-Loop Control

Amazon Robotics wanted to automate the task of placing items from totes into bags and boxes — a high-volume fulfillment step with the potential to save billions through automation. The main bottleneck was that vision-only systems couldn't reliably prevent damage: relying on perception alone, the arm had no way to sense contact during placement, risking harm to the robot, the packaging enclosure, or the picked object itself. I led full-stack development of a work cell that closed this gap by adding force sensing to the loop, taking the system from concept through integration and testing on real hardware.

At the core of the cell is a UR10 robotic arm paired with Zivid point cloud cameras, a Robotiq force/torque sensor, and closed-loop controllers. I designed and fabricated a custom suction-based end effector that integrated mechanical, pneumatic, and electrical subsystems, and developed the eye-to-hand calibration between the end effector and the cameras so the arm could act accurately on what it saw.

For perception, I built a computer-vision program using YOLO, OpenCV, and the Zivid cameras to recognize objects and enclosures in cluttered totes and compute their 3D poses relative to the arm's base. The controls side is where the system really came together: I implemented a behavior-tree force controller with closed-loop feedback for placement operations, backed by a dynamic end-effector model with force-sensor compensation calibration. Rather than relying on position alone, the arm senses contact forces during placement and adjusts in real time — letting it set objects down gently and reliably despite variation in object size, weight, and pose, without damaging the object, the enclosure, or itself.