Clipify - Student Engineering Journal
Clipify grew from an earlier project called ChestCam, which lived in a separate git repository. Both projects explore whether an iPhone can use on-device signals to suggest candidate moments without pretending it knows what mattered to a person.
The design goal is user-controlled: process signals locally where possible, save selected candidate moments instead of everything, and let the user decide what to keep or review. The longer-term idea is a wearable camera that could help organize parts of everyday life and eventually build a short daily video log. The current iPhone app only tests pieces of that idea. The automatic daily video story is not finished.
These are selected notes from the earlier ChestCam V1 history and the newer Clipify V2 history. I grouped related commits when they were part of the same engineering problem.
| Date | Area | Engineering journal |
|---|---|---|
| 2025-04-25 | ChestCam V1 foundation | I started by asking how a camera could suggest that a moment might be worth keeping. I explored ambient capture, device signals, user patterns, and candidate moments instead of simply recording everything. I tried to build too much of the idea at once, which made it hard to tell which part owned each decision. |
| 2025-04 to 2025-06 | Making sensing visible | Early ChestCam used a capture control inspired by the Siri and HomePod animation. It was still a button I could press to capture a moment, but it also showed the detector activity around it. Each detector had its own color, and its size and brightness changed with confidence. The whole button grew with the combined capture-likelihood score. I wanted the interface to show which signals were active and how strongly they were responding instead of hiding everything behind one number. |
| 2025-06-03 | Making signals comparable | Face, sound, motion, scene, place, object, and emotion detectors all produced different kinds of values. I normalized them after finding that some signals dominated the score because of scale rather than meaning. |
| 2025-06-04 | Moment scoring | I combined detector confidence, personal preference, rarity, and decay into a moment score. A face or loud sound was not automatically important, and older or repeated evidence should not stay equally strong forever, so I treated the score as uncertain evidence rather than an answer. |
| 2025-06-06 to 2025-06-07 | Audio and auto capture | Adding audio and Live Photo export caused feedback, routing, and audio-session problems. I also added auto capture with a visible threshold so the user could make the system less or more sensitive. |
| 2025-06-08 to 2025-06-10 | Context-specific preferences | I added persistent profiles and started selecting them from place, scene, and activity signals. One global preference history felt too simple because the same signal could matter differently at home, outdoors, or around different people. |
| 2025-06-11 to 2025-06-13 | Capture context and metadata ordering | I attached timestamps, capture type, detector snapshots, and asset identifiers to saved media. Rapid captures sometimes saved before their metadata was ready, so I changed the flow to finish the snapshot first. |
| 2025-06-13 | Activity explanation experiment | I combined images and detector metadata for on-device activity inference. The first versions had JSON and parsing problems, so I added ways to inspect the input and output instead of trusting a fluent label by itself. |
| 2025-06-17 | Continuous buffer | ChestCam recorded short segments into a bounded rotating buffer. This made it possible to preserve the seconds that had just happened without saving one endless recording. |
| 2025-06-18 to 2025-06-20 | Lifecycle and privacy boundaries | Permission checks and background transitions exposed duplicate saves and broken recovery paths. I also hid sensitive consoles and saved content while the device was locked. Fast capture did not mean every part of the app should be visible. |
| 2025-06-21 to 2025-06-24 | Sensitivity and simulation | I experimented with adjusting sensitivity after long quiet periods and added a simulation path that could trigger capture logic without saving every result. Both changes helped me test behavior without treating each experiment as a real user decision. |
| 2025-06-26 to 2025-07-01 | Performance and detector registry | I moved detectors toward concurrent processing, reused Vision and GPU resources, and introduced a registry for detector names, categories, timing, ordering, and defaults. Performance work quickly became an architecture problem. |
| 2025-06-28 | Field-testing capture likelihood | I took ChestCam on a walk from Bellevue Downtown Park through Meydenbauer Bay Park and Vuecrest and back, watching the capture-likelihood score in real time. Motion could make the score spike and cross the capture threshold, while the crowd talking and laughter around Meydenbauer Bay Park gave me a useful contrast between activity the phone could detect and moments I might actually want to keep. I saved a short field-test video from the walk. |
| 2025-07-01 to 2025-07-04 | Splitting the awareness system | I separated profile, live-update, inference, camera, audio, and lifecycle responsibilities. The useful change was deciding which part owned learned preferences, current detector values, and activity interpretation. |
| 2025-07-08 to 2025-07-12 | Save, audio, and overlay ownership | I created a dedicated save manager after duplicate saves and stale flags kept returning. I also worked through audio coexistence and stopped the camera from restarting while settings or another overlay was visible. |
| 2025-07 to 2025-12 | Real-device capture tuning | I repeatedly tested ChestCam on my own iPhone and adjusted the capture threshold and scoring. The system became more predictable in relatively static scenes, where learned preference, signal strength, and detector confidence stayed stable long enough to influence the score. Moving scenes were harder because the signals changed quickly and could make the capture likelihood jump. This showed me that tuning the score until it worked in one kind of scene was different from understanding when a moment actually mattered. |
| 2026-01 to 2026-03 | Rethinking the product | The preserved history is incomplete for this period. When I returned to the idea, feedback from my mom pushed me to think less about detection by itself and more about how someone would review and use moments in everyday life. |
| 2026-03-29 | Clipify V2 rewrite | I started a new repository and rewrote the app as Clipify instead of renaming ChestCam inside the same history. I carried forward the core question but changed the architecture and product experience. The earlier animated control stayed with me as a design idea: capture should show which signals are active and keep the final choice with the user. |
| 2026-04-02 to 2026-04-10 | Candidate moments and visible uncertainty | Auto capture returned as candidate-moment capture with explicit controls and haptic feedback. I also showed signal confidence in the interface so detector labels would not look like facts. |
| 2026-04-11 to 2026-04-16 | Activity descriptions and lock-state safety | I experimented with Foundation Models and multimodal input to turn sensor evidence into readable activity descriptions. I also added lock-state handling so private history and controls stayed unavailable until protected data could be accessed. |
| 2026-04-28 to 2026-05-03 | Clearer manager and save boundaries | I replaced the growing awareness manager with smaller coordination, inference, profile, and update components. Saving also moved into a dedicated manager because candidate clips, snapshots, metadata, and Live Photo behavior had too many failure cases to stay inside camera callbacks. |
| 2026-05-10 to 2026-05-17 | Key moments and history | I added a bounded key-moment buffer, a review view, local persistence, and history. This moved the project from continuous-capture infrastructure toward a user-reviewable list of candidate moments. |
| 2026-05-24 to 2026-05-26 | Episodes and review-first design | I added episode models to group nearby key moments and replaced an older debugging view with a gallery focused on what the system kept. The groupings were suggestions that the user could inspect, not proof that the app understood the day. |
| 2026-06-04 to 2026-06-07 | Episode history and diary rules | I expanded episode history and added explicit diary eligibility rules with tests. Not every captured moment should enter a daily log, so the rule had to be visible in code instead of hidden in a prompt. |
| 2026-06-08 to 2026-06-20 | Diary coordination and status | I connected episode inference with diary building, added a build ledger, and created clearer waiting, building, complete, and failed states. The hard part was deciding when the data was ready and avoiding duplicate background work after interruptions. |
| 2026-07-03 to 2026-07-04 | Capture ownership and background stability | I added one coordinator to decide who owned the camera engine and strengthened background recovery. This still needs real-device validation, but the code moved away from scattered restart fixes and toward explicit ownership. |
| 2026-07-29 to 2026-07-31 | Turning captures into a reviewable day | I redesigned capture review, grouped nearby captures into suggested events and a day journey, and made diary finalization, playback, and navigation more reliable. I also added a simple trend showing how much of the kept history came from approved automatic captures. A lot of smaller fixes to swipes, scrolling, thumbnails, duplicate events, and camera controls were needed before the experience felt predictable. |
| 2026-08-01 to 2026-08-02 | Making diary journeys portable and more personal | I changed how Clipify writes a full-day diary so long days are summarized in smaller stages instead of failing when too much evidence is sent to the model at once. I also synced the day journey between devices without trying to share device-only photo identifiers, added writing-style choices, and made the final video feel more like a story with animated cards and transitions. The pieces now connect better, but cross-device media and long-day generation still need real-device testing. |
| 2026-08-08 to 2026-08-09 | Choosing stronger diary stories and recovering interrupted reviews | I changed the diary builder so it does not treat every eligible capture as an equally good story beat. It now narrows a day to a bounded set of candidates, uses event context plus simple image-usability and composition signals, removes near-duplicates, and assigns roles such as opener, turn, and close before composing the video. I also redesigned the overlays into more editorial layouts and preserved the diary's original day when saving it. Separately, I fixed a review failure where staged media could survive an interruption but its Core Data record could be missing; Clipify can now recover from the staged metadata instead of leaving the review spinning, with tests for the recovery path. I still need more real-device days before I can judge whether the automatic edits actually make better stories. |
| 2026-08-10 | Making Journey clearer and recovering captures earlier | I finished replacing the old Gallery wording with Journey and made the day view account for captures, episode metadata, and diary days together. More importantly, I now save a small recovery record as soon as a capture is reserved. If the app is interrupted before the full context snapshot finishes, the pending capture still has enough information to return to private review instead of becoming an orphaned file. This improves a failure path, but I still need longer real-device use to know how often it matters in normal capture sessions. |
| 2026-08-11 | Turning Journey into a private Journal | I changed Clipify 1.0 so automatic captures can stay in Clipify's private library instead of going straight into Photos, while user-initiated captures can still be saved to Photos. I also replaced the older live cross-device sync direction with explicit iCloud backup and restore using stable capture identifiers, and added safer backup error handling. On the product side, I renamed the day experience to Journal, made Video Diary more prominent, kept pending review visible inside the selected day, and improved thumbnail preparation and loading. This feels closer to the product I want: a private daily record instead of a camera gallery. Backup/restore and longer real-device days still need more testing. |
Looking back
ChestCam helped me explore whether software could suggest moments from signals and patterns. Clipify pushed the idea toward an everyday, user-controlled review experience. Rewriting the app taught me that carrying forward an idea does not always mean carrying forward the original code.
The hardest parts were not always the visible features. A lot of the work was deciding who owned the camera, microphone, saving, learned state, background recovery, and privacy boundaries. The app is still experimental, especially around camera lifecycle, diary behavior, and episode quality on real devices.
Building Clipify also made me realize that I had spent more time studying how a phone detects a moment than how a person chooses one. Someone may press the shutter because of timing, composition, light, or something that is difficult to turn into a score. I requested Photography for senior year because I want to understand that decision better and question what Clipify measures, not just add more detectors.