Clipify - Student Engineering Journal
Clipify grew from an earlier project called ChestCam, which lived in a separate git repository. Both projects explore whether an iPhone can use on-device signals to suggest candidate moments without pretending it knows what mattered to a person.
The design goal is user-controlled: process signals locally where possible, save selected candidate moments instead of everything, and let the user decide what to keep or review. The longer-term idea is a wearable camera that could help organize parts of everyday life and eventually build a short daily video log. The current iPhone app only tests pieces of that idea. The automatic daily video story is not finished.
These are selected notes from the earlier ChestCam V1 history and the newer Clipify V2 history. I grouped related commits when they were part of the same engineering problem.
| Date | Area | Engineering journal |
|---|---|---|
| 2025-04-25 | ChestCam V1 foundation | I started by asking how a camera could suggest that a moment might be worth keeping. I explored ambient capture, device signals, user patterns, and candidate moments instead of simply recording everything. I tried to build too much of the idea at once, which made it hard to tell which part owned each decision. |
| 2025-04 to 2025-06 | Making sensing visible | Early ChestCam used a capture control inspired by the Siri and HomePod animation. It was still a button I could press to capture a moment, but it also showed the detector activity around it. Each detector had its own color, and its size and brightness changed with confidence. The whole button grew with the combined capture-likelihood score. I wanted the interface to show which signals were active and how strongly they were responding instead of hiding everything behind one number. |
| 2025-06-03 | Making signals comparable | Face, sound, motion, scene, place, object, and emotion detectors all produced different kinds of values. I normalized them after finding that some signals dominated the score because of scale rather than meaning. |
| 2025-06-04 | Moment scoring | I combined detector confidence, personal preference, rarity, and decay into a moment score. A face or loud sound was not automatically important, and older or repeated evidence should not stay equally strong forever, so I treated the score as uncertain evidence rather than an answer. |
| 2025-06-06 to 2025-06-07 | Audio and auto capture | Adding audio and Live Photo export caused feedback, routing, and audio-session problems. I also added auto capture with a visible threshold so the user could make the system less or more sensitive. |
| 2025-06-08 to 2025-06-10 | Context-specific preferences | I added persistent profiles and started selecting them from place, scene, and activity signals. One global preference history felt too simple because the same signal could matter differently at home, outdoors, or around different people. |
| 2025-06-11 to 2025-06-13 | Capture context and metadata ordering | I attached timestamps, capture type, detector snapshots, and asset identifiers to saved media. Rapid captures sometimes saved before their metadata was ready, so I changed the flow to finish the snapshot first. |
| 2025-06-13 | Activity explanation experiment | I combined images and detector metadata for on-device activity inference. The first versions had JSON and parsing problems, so I added ways to inspect the input and output instead of trusting a fluent label by itself. |
| 2025-06-17 | Continuous buffer | ChestCam recorded short segments into a bounded rotating buffer. This made it possible to preserve the seconds that had just happened without saving one endless recording. |
| 2025-06-18 to 2025-06-20 | Lifecycle and privacy boundaries | Permission checks and background transitions exposed duplicate saves and broken recovery paths. I also hid sensitive consoles and saved content while the device was locked. Fast capture did not mean every part of the app should be visible. |
| 2025-06-21 to 2025-06-24 | Sensitivity and simulation | I experimented with adjusting sensitivity after long quiet periods and added a simulation path that could trigger capture logic without saving every result. Both changes helped me test behavior without treating each experiment as a real user decision. |
| 2025-06-26 to 2025-07-01 | Performance and detector registry | I moved detectors toward concurrent processing, reused Vision and GPU resources, and introduced a registry for detector names, categories, timing, ordering, and defaults. Performance work quickly became an architecture problem. |
| 2025-06-28 | Field-testing capture likelihood | I took ChestCam on a walk from Bellevue Downtown Park through Meydenbauer Bay Park and Vuecrest and back, watching the capture-likelihood score in real time. Motion could make the score spike and cross the capture threshold, while the crowd talking and laughter around Meydenbauer Bay Park gave me a useful contrast between activity the phone could detect and moments I might actually want to keep. I saved a short field-test video from the walk. |
| 2025-07-01 to 2025-07-04 | Splitting the awareness system | I separated profile, live-update, inference, camera, audio, and lifecycle responsibilities. The useful change was deciding which part owned learned preferences, current detector values, and activity interpretation. |
| 2025-07-08 to 2025-07-12 | Save, audio, and overlay ownership | I created a dedicated save manager after duplicate saves and stale flags kept returning. I also worked through audio coexistence and stopped the camera from restarting while settings or another overlay was visible. |
| 2025-07 to 2025-12 | Real-device capture tuning | I repeatedly tested ChestCam on my own iPhone and adjusted the capture threshold and scoring. The system became more predictable in relatively static scenes, where learned preference, signal strength, and detector confidence stayed stable long enough to influence the score. Moving scenes were harder because the signals changed quickly and could make the capture likelihood jump. This showed me that tuning the score until it worked in one kind of scene was different from understanding when a moment actually mattered. |
| 2026-01 to 2026-03 | Rethinking the product | The preserved history is incomplete for this period. When I returned to the idea, feedback from my mom pushed me to think less about detection by itself and more about how someone would review and use moments in everyday life. |
| 2026-03-29 | Clipify V2 rewrite | I started a new repository and rewrote the app as Clipify instead of renaming ChestCam inside the same history. I carried forward the core question but changed the architecture and product experience. The earlier animated control stayed with me as a design idea: capture should show which signals are active and keep the final choice with the user. |
| 2026-04-02 to 2026-04-10 | Candidate moments and visible uncertainty | Auto capture returned as candidate-moment capture with explicit controls and haptic feedback. I also showed signal confidence in the interface so detector labels would not look like facts. |
| 2026-04-11 to 2026-04-16 | Activity descriptions and lock-state safety | I experimented with Foundation Models and multimodal input to turn sensor evidence into readable activity descriptions. I also added lock-state handling so private history and controls stayed unavailable until protected data could be accessed. |
| 2026-04-28 to 2026-05-03 | Clearer manager and save boundaries | I replaced the growing awareness manager with smaller coordination, inference, profile, and update components. Saving also moved into a dedicated manager because candidate clips, snapshots, metadata, and Live Photo behavior had too many failure cases to stay inside camera callbacks. |
| 2026-05-10 to 2026-05-17 | Key moments and history | I added a bounded key-moment buffer, a review view, local persistence, and history. This moved the project from continuous-capture infrastructure toward a user-reviewable list of candidate moments. |
| 2026-05-24 to 2026-05-26 | Episodes and review-first design | I added episode models to group nearby key moments and replaced an older debugging view with a gallery focused on what the system kept. The groupings were suggestions that the user could inspect, not proof that the app understood the day. |
| 2026-06-04 to 2026-06-07 | Episode history and diary rules | I expanded episode history and added explicit diary eligibility rules with tests. Not every captured moment should enter a daily log, so the rule had to be visible in code instead of hidden in a prompt. |
| 2026-06-08 to 2026-06-20 | Diary coordination and status | I connected episode inference with diary building, added a build ledger, and created clearer waiting, building, complete, and failed states. The hard part was deciding when the data was ready and avoiding duplicate background work after interruptions. |
| 2026-07-03 to 2026-07-04 | Capture ownership and background stability | I added one coordinator to decide who owned the camera engine and strengthened background recovery. This still needs real-device validation, but the code moved away from scattered restart fixes and toward explicit ownership. |
| 2026-07-29 to 2026-07-31 | Turning captures into a reviewable day | I redesigned capture review, grouped nearby captures into suggested events and a day journey, and made diary finalization, playback, and navigation more reliable. I also added a simple trend showing how much of the kept history came from approved automatic captures. A lot of smaller fixes to swipes, scrolling, thumbnails, duplicate events, and camera controls were needed before the experience felt predictable. |
| 2026-08-01 to 2026-08-02 | Making diary journeys portable and more personal | I changed how Clipify writes a full-day diary so long days are summarized in smaller stages instead of failing when too much evidence is sent to the model at once. I also synced the day journey between devices without trying to share device-only photo identifiers, added writing-style choices, and made the final video feel more like a story with animated cards and transitions. The pieces now connect better, but cross-device media and long-day generation still need real-device testing. |
| 2026-08-08 to 2026-08-09 | Choosing stronger diary stories and recovering interrupted reviews | I changed the diary builder so it does not treat every eligible capture as an equally good story beat. It now narrows a day to a bounded set of candidates, uses event context plus simple image-usability and composition signals, removes near-duplicates, and assigns roles such as opener, turn, and close before composing the video. I also redesigned the overlays into more editorial layouts and preserved the diary's original day when saving it. Separately, I fixed a review failure where staged media could survive an interruption but its Core Data record could be missing; Clipify can now recover from the staged metadata instead of leaving the review spinning, with tests for the recovery path. I still need more real-device days before I can judge whether the automatic edits actually make better stories. |
| 2026-08-10 | Making Journey clearer and recovering captures earlier | I finished replacing the old Gallery wording with Journey and made the day view account for captures, episode metadata, and diary days together. More importantly, I now save a small recovery record as soon as a capture is reserved. If the app is interrupted before the full context snapshot finishes, the pending capture still has enough information to return to private review instead of becoming an orphaned file. This improves a failure path, but I still need longer real-device use to know how often it matters in normal capture sessions. |
| 2026-08-11 | Turning Journey into a private Journal | I changed Clipify 1.0 so automatic captures can stay in Clipify's private library instead of going straight into Photos, while user-initiated captures can still be saved to Photos. I also replaced the older live cross-device sync direction with explicit iCloud backup and restore using stable capture identifiers, and added safer backup error handling. On the product side, I renamed the day experience to Journal, made Video Diary more prominent, kept pending review visible inside the selected day, and improved thumbnail preparation and loading. This feels closer to the product I want: a private daily record instead of a camera gallery. Backup/restore and longer real-device days still need more testing. |
| 2026-08-12 | Giving private media a lifecycle | Making automatic captures private created a new question: what should happen to those files later? I added choices to keep private media forever or clean it up after 15, 30, or 90 days, plus actions to save a private capture to Photos or delete it from Clipify. I also made the Journal load a smaller window around the day being viewed, cancel image work that is no longer needed, and keep review items visible more reliably. The save-to-Photos path now records its recovery link before the private copy is removed, so the connection can survive an interruption. This gives private media an explicit lifecycle instead of just hiding it from Photos. I still need longer real-device testing for cleanup, backup retries, and interruption recovery. |
| 2026-08-15 | Making recovery and permissions more trustworthy | After making captures private, I found there were still ways cleanup, restore, or app transitions could leave the user unsure what happened. I changed restore so it previews the backup before merging and reports failures, made private-media deletion reconcile the Journal before removing the file and keep failed cleanups retriable, stopped asking for notification permission just because the background schedule existed, and made capture resume wait until the app, camera ownership, and overlays are all ready. I added tests around the cleanup and lifecycle rules. This is recovery and product-trust work; I still need longer real-device days to know how often these paths matter. |
| 2026-08-19 | Making private data actually erasable | Moving automatic captures into a private Journal made me look harder at what “private” meant outside the normal UI. I moved the rolling capture buffer out of Documents into protected app storage, turned off file sharing and open-in-place exposure, and added an Erase Clipify Data flow that clears Journal and history stores, learned preferences, inference traces, private captures, save receipts, and related local state, with an option to include Clipify's iCloud backup. I added tests around the deletion path and kept onboarding state separate from user data when needed. This closes privacy and data-ownership gaps in the implementation, but I still need real-device testing around interrupted deletion and backup removal before treating the path as fully validated. |
| 2026-08-22 to 2026-08-23 | Making failed captures disappear cleanly | I found that a recording could fail after Clipify had already created a key-moment record, which meant a moment with no durable media could keep moving through inference, Journal grouping, or cloud sync. I added one failure-cleanup path that cancels the in-flight processing, removes the failed moment and its inference attachments and traces, waits to sync media-backed moments until the Photos link exists, and rebuilds the affected day's episodes so deleted moments do not leave stale stories behind. I also serialized camera ownership when scenes hand off the shared recorder and made erase and restore paths surface storage failures instead of silently continuing. Tests cover the failed-capture cleanup and reconciliation rules. This makes the capture lifecycle more consistent in code, but I still need longer real-device sessions and interrupted-recording tests before calling it reliable. |
| 2026-08-27 | Letting a moment I chose survive a failed inference | I found a bad case in the Journal pipeline: a moment I captured myself, or explicitly chose to keep, could still disappear from the day story if automatic episode inference failed or had not grouped it yet. I changed Journal and Journey regeneration so user-chosen moments stay eligible even without a settled episode, and added standalone captions that only use reviewed or high-confidence activity and setting evidence; otherwise they simply say it was a moment I chose to capture or keep. I also moved those editorial overlays into the exported video path so the saved Journey includes the same evidence-grounded context. Tests cover manual, kept, and low-confidence cases. This protects user intent better, but it does not prove the automatic diary story is good; I still need more real-device days. |
| 2026-08-28 | Keeping edits and private-media actions from leaving half-saved state | I found several places where a user action could change one copy of state before another write finished. I made profile renames and preference-feedback writes atomic, stopped deleting a regular capture from undoing learned preference unless that capture had actually been favorited, repaired surviving episode assignments when a deleted moment was still referenced by an older batch, and blocked competing save-to-Photos, delete, or diary-share preparation on the same private media. I also made busy iCloud restores explain how to retry. Tests cover the persistence, reconciliation, and one-action-at-a-time rules. This makes these failure paths more consistent in code, but I still need longer real-device and interrupted-operation testing before calling the editing and restore flows reliable. |
| 2026-08-29 | Making failures easier to recover from and test | I found that some failure paths still protected the data but left the person with a dead end or made the failure hard to reproduce. I added retry-oriented messages for failed gallery video loads and diary sharing, gave the main capture control clearer VoiceOver label, value, and actions, added injected write-failure tests to verify existing profile data survives a failed save, and made diary-share preparation single-flight so two exports cannot race. I also added repository quality gates that require risky changes to document failure, recovery, validation, and rollback. Tests cover the new failure paths and accessibility semantics. This makes recovery behavior easier to inspect and test, but it does not mean Clipify is broadly reliable or fully accessible; I still need real-device VoiceOver and interrupted share/gallery testing. |
| 2026-08-30 to 2026-08-31 | Letting reviewed moments refresh the diary | I noticed that reviewing a key moment could update the evidence I saw in Journal without rebuilding an existing video diary, so the saved story could keep using older context. I added a delayed per-day refresh after reviewed moments, made the pending refresh survive an app restart, and kept the old diary until a replacement is safely created. If regeneration fails, the day stays pending instead of pretending the update succeeded. Tests cover pending refresh storage, replacement detection, and repeated edits. This makes reviewed corrections more likely to reach the generated diary in code, but I still need interrupted and real-device diary tests before I know the flow is reliable. |
| 2026-09-03 to 2026-09-05 | Making the video diary recoverable without guessing | I found that a video diary could fail, disappear from Photos, or return after restore while Clipify still remembered a different state. I made failed builds show as failures instead of looking unfinished, reconciled a missing Photos diary only when full Photos access made that absence meaningful, added explicit save-another-copy and retry paths that keep the original safe, and made restore relink a diary only when there is exactly one unambiguous Clipify diary for that logical day. I also stopped logging APNs tokens and generated inference content. Tests cover the failure, deletion, copy, retry, and ambiguous-restore cases. This makes the diary lifecycle more honest and recoverable in code, but I still need longer real-device and interrupted restore/save/share testing before calling it reliable. |
| 2026-09-07 to 2026-09-08 | Letting me choose what belongs in the diary | I realized that keeping a moment and putting it in the video diary are related but not the same decision. I added an explicit choice to let Clipify choose, always include, or leave a moment out, and I made manual captures survive the automatic selection limits unless I exclude them. I also stopped later review feedback from silently replacing a video diary I may already have watched or shared; the app now waits for an explicit Update diary action. Finally, I changed inference writes so newer review choices are preserved when older background work finishes. Tests cover manual inclusion, explicit include/exclude choices, saved-diary updates, and stale inference writes. This gives me more control over the final story in code, but it does not prove the diary is good or that the flow is reliable on every device; I still need longer real-device testing. |
Looking back
ChestCam helped me explore whether software could suggest moments from signals and patterns. Clipify pushed the idea toward an everyday, user-controlled review experience. Rewriting the app taught me that carrying forward an idea does not always mean carrying forward the original code.
The hardest parts were not always the visible features. A lot of the work was deciding who owned the camera, microphone, saving, learned state, background recovery, and privacy boundaries. The app is still experimental, especially around camera lifecycle, diary behavior, and episode quality on real devices.
Building Clipify also made me realize that I had spent more time studying how a phone detects a moment than how a person chooses one. Someone may press the shutter because of timing, composition, light, or something that is difficult to turn into a score. I requested Photography for senior year because I want to understand that decision better and question what Clipify measures, not just add more detectors.