Les Choses qui s'en Vont (Musée de Poche)
A large format projected installation for the final room of the exhibition. Two illustrations from the book come alive under the children's gestures, captured with no controller and no touchscreen: the body is the only interface.
Context & Challenges
- Alive even with no audience: the installation had to run continuously in the room, image and soundtrack, behaving as a self sufficient piece of media. Interaction is not a prerequisite but a bonus: it wakes up when a child steps into frame.
- Extending the gesture of the book: in the album, the child turns a tracing sheet to animate the image. The installation had to carry that principle over to the scale of the body, without adding anything foreign to the work.
- An audience that does not read instructions: young children facing a projection with no button and no screen. Everything had to be understood by watching others do it.
- Respecting a strong graphic world: Beatrice Alemagna's drawings are made with pencil, pen and brush. Any visible digital intervention would have betrayed the material.
- Bodies too small for the sensor: a depth camera partly infers distance from the apparent size of a body, so a child reads as a more distant adult. Add to that bodies that move fast and are rarely alone in frame.
Engineering & Method
- Artwork preparation: cleaning the album scans and creating the few animation frames needed, staying as close as possible to the original line.
- Contactless capture: an Intel RealSense camera and the Nuitrack API for skeletons, with nothing worn or touched by the child. The rig is three elements: a projector, a projection surface, and the camera sitting at the foot of the screen.
- Scene logic in Unity: two tableaux alternating automatically, each with its own interaction rules and sound response.
- Continuous soundscape: birdsong at rest, wingbeats on takeoff, rainfall then wind as the rain moves away.
Technical Breakdown
1. Making skeleton tracking hold on children's bodies
Challenge: a depth camera partly infers distance from the apparent size of a body. Children being smaller than adults, the engine kept placing them too far away and the skeleton dropped out. Add constant movement, several children in frame at once, and sometimes a parent carrying one.
Solution: an Intel RealSense camera with the Nuitrack API, whose detection parameters were reworked until the skeleton held on children's bodies. This is the foundation for everything else: without reliable head and hand joints, neither interaction works.
2. The bird: rewarding stillness
Challenge: convey without a single word that you do not catch a bird by running after it.
Solution: the bird flies freely between perches. When a child reaches out, hand higher than the other, and holds the pose for about two seconds, the bird comes to land on it. Conversely, the group's agitation feeds a nervousness counter: past a threshold the bird takes off, and sharp movements interrupt its song. The installation rewards slowness, which is rare in a piece for children.
3. The rain: building a shelter with your body
Challenge: give the child a physical hold on something falling from the sky, without handing them a button to stop it.
Solution: the same nervousness counter brings the rain in, curtain after curtain, and the mass of water follows the group's average movement. Once the downpour is established, an invisible volume locks onto each child's head: raising both hands above your head cuts the drops passing through it. You do not command the rain, you make yourself a roof with your arms.
4. A hand drawn instruction panel
Challenge: explain two gestures to an audience that cannot read yet, without putting text on the projection or breaking the album's world with standard pictograms.
Solution: a panel drawn entirely by hand, imitating Beatrice Alemagna's line as closely as possible, then laser engraved. Four vignettes, two per scene, show the gesture and its effect. No words are needed, and the signage belongs to the same world as the projected images.
5. The installation running
Note: the installation was not documented in operation, as filming children in the gallery raises image rights questions. The sequence below is recorded from the real time engine, with its soundtrack.
Illustrations: Beatrice Alemagna, from the album Les choses qui s'en vont.
Exhibition: conceived and produced by the Musée de poche.
Installation development: Sébastien Lebon, at Crafteke studio.
Note: visuals rendered from the real time engine, the installation not having been documented in operation.
Exhibition: conceived and produced by the Musée de poche.
Installation development: Sébastien Lebon, at Crafteke studio.
Note: visuals rendered from the real time engine, the installation not having been documented in operation.