Clipify - Student Engineering Journal

Clipify grew from an earlier project called ChestCam, which lived in a separate git repository. Both projects explore whether an iPhone can use on-device signals to suggest candidate moments without pretending it knows what mattered to a person.

The design goal is user-controlled: process signals locally where possible, save selected candidate moments instead of everything, and let the user decide what to keep or review. The longer-term idea is a wearable camera that could help organize parts of everyday life and eventually build a short daily video log. The current iPhone app only tests pieces of that idea. The automatic daily video story is not finished.

These are selected notes from the earlier ChestCam V1 history and the newer Clipify V2 history. I grouped related commits when they were part of the same engineering problem.

DateAreaEngineering journal
2025-04-25ChestCam V1 foundationI started by asking how a camera could suggest that a moment might be worth keeping. I explored ambient capture, device signals, user patterns, and candidate moments instead of simply recording everything. I tried to build too much of the idea at once, which made it hard to tell which part owned each decision.
2025-04 to 2025-06Making sensing visibleEarly ChestCam used a capture control inspired by the Siri and HomePod animation. It was still a button I could press to capture a moment, but it also showed the detector activity around it. Each detector had its own color, and its size and brightness changed with confidence. The whole button grew with the combined capture-likelihood score. I wanted the interface to show which signals were active and how strongly they were responding instead of hiding everything behind one number.
2025-06-03Making signals comparableFace, sound, motion, scene, place, object, and emotion detectors all produced different kinds of values. I normalized them after finding that some signals dominated the score because of scale rather than meaning.
2025-06-04Moment scoringI combined detector confidence, personal preference, rarity, and decay into a moment score. A face or loud sound was not automatically important, and older or repeated evidence should not stay equally strong forever, so I treated the score as uncertain evidence rather than an answer.
2025-06-06 to 2025-06-07Audio and auto captureAdding audio and Live Photo export caused feedback, routing, and audio-session problems. I also added auto capture with a visible threshold so the user could make the system less or more sensitive.
2025-06-08 to 2025-06-10Context-specific preferencesI added persistent profiles and started selecting them from place, scene, and activity signals. One global preference history felt too simple because the same signal could matter differently at home, outdoors, or around different people.
2025-06-11 to 2025-06-13Capture context and metadata orderingI attached timestamps, capture type, detector snapshots, and asset identifiers to saved media. Rapid captures sometimes saved before their metadata was ready, so I changed the flow to finish the snapshot first.
2025-06-13Activity explanation experimentI combined images and detector metadata for on-device activity inference. The first versions had JSON and parsing problems, so I added ways to inspect the input and output instead of trusting a fluent label by itself.
2025-06-17Continuous bufferChestCam recorded short segments into a bounded rotating buffer. This made it possible to preserve the seconds that had just happened without saving one endless recording.
2025-06-18 to 2025-06-20Lifecycle and privacy boundariesPermission checks and background transitions exposed duplicate saves and broken recovery paths. I also hid sensitive consoles and saved content while the device was locked. Fast capture did not mean every part of the app should be visible.
2025-06-21 to 2025-06-24Sensitivity and simulationI experimented with adjusting sensitivity after long quiet periods and added a simulation path that could trigger capture logic without saving every result. Both changes helped me test behavior without treating each experiment as a real user decision.
2025-06-26 to 2025-07-01Performance and detector registryI moved detectors toward concurrent processing, reused Vision and GPU resources, and introduced a registry for detector names, categories, timing, ordering, and defaults. Performance work quickly became an architecture problem.
2025-06-28Field-testing capture likelihoodI took ChestCam on a walk from Bellevue Downtown Park through Meydenbauer Bay Park and Vuecrest and back, watching the capture-likelihood score in real time. Motion could make the score spike and cross the capture threshold, while the crowd talking and laughter around Meydenbauer Bay Park gave me a useful contrast between activity the phone could detect and moments I might actually want to keep. I saved a short field-test video from the walk.
2025-07-01 to 2025-07-04Splitting the awareness systemI separated profile, live-update, inference, camera, audio, and lifecycle responsibilities. The useful change was deciding which part owned learned preferences, current detector values, and activity interpretation.
2025-07-08 to 2025-07-12Save, audio, and overlay ownershipI created a dedicated save manager after duplicate saves and stale flags kept returning. I also worked through audio coexistence and stopped the camera from restarting while settings or another overlay was visible.
2025-07 to 2025-12Real-device capture tuningI repeatedly tested ChestCam on my own iPhone and adjusted the capture threshold and scoring. The system became more predictable in relatively static scenes, where learned preference, signal strength, and detector confidence stayed stable long enough to influence the score. Moving scenes were harder because the signals changed quickly and could make the capture likelihood jump. This showed me that tuning the score until it worked in one kind of scene was different from understanding when a moment actually mattered.
2026-01 to 2026-03Rethinking the productThe preserved history is incomplete for this period. When I returned to the idea, feedback from my mom pushed me to think less about detection by itself and more about how someone would review and use moments in everyday life.
2026-03-29Clipify V2 rewriteI started a new repository and rewrote the app as Clipify instead of renaming ChestCam inside the same history. I carried forward the core question but changed the architecture and product experience. The earlier animated control stayed with me as a design idea: capture should show which signals are active and keep the final choice with the user.
2026-04-02 to 2026-04-10Candidate moments and visible uncertaintyAuto capture returned as candidate-moment capture with explicit controls and haptic feedback. I also showed signal confidence in the interface so detector labels would not look like facts.
2026-04-11 to 2026-04-16Activity descriptions and lock-state safetyI experimented with Foundation Models and multimodal input to turn sensor evidence into readable activity descriptions. I also added lock-state handling so private history and controls stayed unavailable until protected data could be accessed.
2026-04-28 to 2026-05-03Clearer manager and save boundariesI replaced the growing awareness manager with smaller coordination, inference, profile, and update components. Saving also moved into a dedicated manager because candidate clips, snapshots, metadata, and Live Photo behavior had too many failure cases to stay inside camera callbacks.
2026-05-10 to 2026-05-17Key moments and historyI added a bounded key-moment buffer, a review view, local persistence, and history. This moved the project from continuous-capture infrastructure toward a user-reviewable list of candidate moments.
2026-05-24 to 2026-05-26Episodes and review-first designI added episode models to group nearby key moments and replaced an older debugging view with a gallery focused on what the system kept. The groupings were suggestions that the user could inspect, not proof that the app understood the day.
2026-06-04 to 2026-06-07Episode history and diary rulesI expanded episode history and added explicit diary eligibility rules with tests. Not every captured moment should enter a daily log, so the rule had to be visible in code instead of hidden in a prompt.
2026-06-08 to 2026-06-20Diary coordination and statusI connected episode inference with diary building, added a build ledger, and created clearer waiting, building, complete, and failed states. The hard part was deciding when the data was ready and avoiding duplicate background work after interruptions.
2026-07-03 to 2026-07-04Capture ownership and background stabilityI added one coordinator to decide who owned the camera engine and strengthened background recovery. This still needs real-device validation, but the code moved away from scattered restart fixes and toward explicit ownership.
2026-07-29 to 2026-07-31Turning captures into a reviewable dayI redesigned capture review, grouped nearby captures into suggested events and a day journey, and made diary finalization, playback, and navigation more reliable. I also added a simple trend showing how much of the kept history came from approved automatic captures. A lot of smaller fixes to swipes, scrolling, thumbnails, duplicate events, and camera controls were needed before the experience felt predictable.
2026-08-01 to 2026-08-02Making diary journeys portable and more personalI changed how Clipify writes a full-day diary so long days are summarized in smaller stages instead of failing when too much evidence is sent to the model at once. I also synced the day journey between devices without trying to share device-only photo identifiers, added writing-style choices, and made the final video feel more like a story with animated cards and transitions. The pieces now connect better, but cross-device media and long-day generation still need real-device testing.
2026-08-08 to 2026-08-09Choosing stronger diary stories and recovering interrupted reviewsI changed the diary builder so it does not treat every eligible capture as an equally good story beat. It now narrows a day to a bounded set of candidates, uses event context plus simple image-usability and composition signals, removes near-duplicates, and assigns roles such as opener, turn, and close before composing the video. I also redesigned the overlays into more editorial layouts and preserved the diary's original day when saving it. Separately, I fixed a review failure where staged media could survive an interruption but its Core Data record could be missing; Clipify can now recover from the staged metadata instead of leaving the review spinning, with tests for the recovery path. I still need more real-device days before I can judge whether the automatic edits actually make better stories.
2026-08-10Making Journey clearer and recovering captures earlierI finished replacing the old Gallery wording with Journey and made the day view account for captures, episode metadata, and diary days together. More importantly, I now save a small recovery record as soon as a capture is reserved. If the app is interrupted before the full context snapshot finishes, the pending capture still has enough information to return to private review instead of becoming an orphaned file. This improves a failure path, but I still need longer real-device use to know how often it matters in normal capture sessions.
2026-08-11Turning Journey into a private JournalI changed Clipify 1.0 so automatic captures can stay in Clipify's private library instead of going straight into Photos, while user-initiated captures can still be saved to Photos. I also replaced the older live cross-device sync direction with explicit iCloud backup and restore using stable capture identifiers, and added safer backup error handling. On the product side, I renamed the day experience to Journal, made Video Diary more prominent, kept pending review visible inside the selected day, and improved thumbnail preparation and loading. This feels closer to the product I want: a private daily record instead of a camera gallery. Backup/restore and longer real-device days still need more testing.
2026-08-12Giving private media a lifecycleMaking automatic captures private created a new question: what should happen to those files later? I added choices to keep private media forever or clean it up after 15, 30, or 90 days, plus actions to save a private capture to Photos or delete it from Clipify. I also made the Journal load a smaller window around the day being viewed, cancel image work that is no longer needed, and keep review items visible more reliably. The save-to-Photos path now records its recovery link before the private copy is removed, so the connection can survive an interruption. This gives private media an explicit lifecycle instead of just hiding it from Photos. I still need longer real-device testing for cleanup, backup retries, and interruption recovery.
2026-08-15Making recovery and permissions more trustworthyAfter making captures private, I found there were still ways cleanup, restore, or app transitions could leave the user unsure what happened. I changed restore so it previews the backup before merging and reports failures, made private-media deletion reconcile the Journal before removing the file and keep failed cleanups retriable, stopped asking for notification permission just because the background schedule existed, and made capture resume wait until the app, camera ownership, and overlays are all ready. I added tests around the cleanup and lifecycle rules. This is recovery and product-trust work; I still need longer real-device days to know how often these paths matter.
2026-08-19Making private data actually erasableMoving automatic captures into a private Journal made me look harder at what “private” meant outside the normal UI. I moved the rolling capture buffer out of Documents into protected app storage, turned off file sharing and open-in-place exposure, and added an Erase Clipify Data flow that clears Journal and history stores, learned preferences, inference traces, private captures, save receipts, and related local state, with an option to include Clipify's iCloud backup. I added tests around the deletion path and kept onboarding state separate from user data when needed. This closes privacy and data-ownership gaps in the implementation, but I still need real-device testing around interrupted deletion and backup removal before treating the path as fully validated.
2026-08-22 to 2026-08-23Making failed captures disappear cleanlyI found that a recording could fail after Clipify had already created a key-moment record, which meant a moment with no durable media could keep moving through inference, Journal grouping, or cloud sync. I added one failure-cleanup path that cancels the in-flight processing, removes the failed moment and its inference attachments and traces, waits to sync media-backed moments until the Photos link exists, and rebuilds the affected day's episodes so deleted moments do not leave stale stories behind. I also serialized camera ownership when scenes hand off the shared recorder and made erase and restore paths surface storage failures instead of silently continuing. Tests cover the failed-capture cleanup and reconciliation rules. This makes the capture lifecycle more consistent in code, but I still need longer real-device sessions and interrupted-recording tests before calling it reliable.
2026-08-27Letting a moment I chose survive a failed inferenceI found a bad case in the Journal pipeline: a moment I captured myself, or explicitly chose to keep, could still disappear from the day story if automatic episode inference failed or had not grouped it yet. I changed Journal and Journey regeneration so user-chosen moments stay eligible even without a settled episode, and added standalone captions that only use reviewed or high-confidence activity and setting evidence; otherwise they simply say it was a moment I chose to capture or keep. I also moved those editorial overlays into the exported video path so the saved Journey includes the same evidence-grounded context. Tests cover manual, kept, and low-confidence cases. This protects user intent better, but it does not prove the automatic diary story is good; I still need more real-device days.
2026-08-28Keeping edits and private-media actions from leaving half-saved stateI found several places where a user action could change one copy of state before another write finished. I made profile renames and preference-feedback writes atomic, stopped deleting a regular capture from undoing learned preference unless that capture had actually been favorited, repaired surviving episode assignments when a deleted moment was still referenced by an older batch, and blocked competing save-to-Photos, delete, or diary-share preparation on the same private media. I also made busy iCloud restores explain how to retry. Tests cover the persistence, reconciliation, and one-action-at-a-time rules. This makes these failure paths more consistent in code, but I still need longer real-device and interrupted-operation testing before calling the editing and restore flows reliable.
2026-08-29Making failures easier to recover from and testI found that some failure paths still protected the data but left the person with a dead end or made the failure hard to reproduce. I added retry-oriented messages for failed gallery video loads and diary sharing, gave the main capture control clearer VoiceOver label, value, and actions, added injected write-failure tests to verify existing profile data survives a failed save, and made diary-share preparation single-flight so two exports cannot race. I also added repository quality gates that require risky changes to document failure, recovery, validation, and rollback. Tests cover the new failure paths and accessibility semantics. This makes recovery behavior easier to inspect and test, but it does not mean Clipify is broadly reliable or fully accessible; I still need real-device VoiceOver and interrupted share/gallery testing.
2026-08-30 to 2026-08-31Letting reviewed moments refresh the diaryI noticed that reviewing a key moment could update the evidence I saw in Journal without rebuilding an existing video diary, so the saved story could keep using older context. I added a delayed per-day refresh after reviewed moments, made the pending refresh survive an app restart, and kept the old diary until a replacement is safely created. If regeneration fails, the day stays pending instead of pretending the update succeeded. Tests cover pending refresh storage, replacement detection, and repeated edits. This makes reviewed corrections more likely to reach the generated diary in code, but I still need interrupted and real-device diary tests before I know the flow is reliable.
2026-09-03 to 2026-09-05Making the video diary recoverable without guessingI found that a video diary could fail, disappear from Photos, or return after restore while Clipify still remembered a different state. I made failed builds show as failures instead of looking unfinished, reconciled a missing Photos diary only when full Photos access made that absence meaningful, added explicit save-another-copy and retry paths that keep the original safe, and made restore relink a diary only when there is exactly one unambiguous Clipify diary for that logical day. I also stopped logging APNs tokens and generated inference content. Tests cover the failure, deletion, copy, retry, and ambiguous-restore cases. This makes the diary lifecycle more honest and recoverable in code, but I still need longer real-device and interrupted restore/save/share testing before calling it reliable.
2026-09-07 to 2026-09-08Letting me choose what belongs in the diaryI realized that keeping a moment and putting it in the video diary are related but not the same decision. I added an explicit choice to let Clipify choose, always include, or leave a moment out, and I made manual captures survive the automatic selection limits unless I exclude them. I also stopped later review feedback from silently replacing a video diary I may already have watched or shared; the app now waits for an explicit Update diary action. Finally, I changed inference writes so newer review choices are preserved when older background work finishes. Tests cover manual inclusion, explicit include/exclude choices, saved-diary updates, and stale inference writes. This gives me more control over the final story in code, but it does not prove the diary is good or that the flow is reliable on every device; I still need longer real-device testing.
2026-11-02Keeping community learning optional and reviewableI enabled a separate Share Review Feedback path and made community learning updates visible in Settings. Someone can receive signed, reviewed activity and setting knowledge without sharing feedback, while contributions leave out media, coordinates, profile and place names, free text, filenames, and asset IDs. Repeated answers from one account do not add extra weight, and a candidate needs at least 20 agreeing accounts with 80% agreement before human review and publication. Twelve knowledge-base tests and the offline signing pipeline passed, and a Release build plus the update screen were checked on a physical iPhone. No learned package has been published, cross-account CloudKit permissions and a live signed update still need testing, and this is not a TestFlight or App Store release.
2026-11-02Updating a diary without replacing it with incomplete mediaApplying capture feedback could reach the video rebuild even when some original media was missing or could not be opened. I changed Update diary to use the current reviewed captures without rewriting the journal story, require every selected source before replacing the video, and keep the existing diary when loading or building fails. The app now explains the failure and offers Try Again. Focused tests for missing sources, no journal mutation, and failure presentation passed in a simulator. The full validation run still failed on two separate keyframe-selection checks, and no physical-device diary update is recorded. This is source code only and not a TestFlight or App Store release.
2026-11-02Putting help and recovery guidance inside ClipifyThe app's behavior had grown beyond what someone could infer from the controls, especially around when capture has started, why a diary leaves clips out, and what happens when original media is unavailable. I added a Support area in Settings with plain-language explanations, troubleshooting steps, a link to iOS permission settings, and a contact path that sends nothing automatically. The guidance warns against deleting the app as a recovery step because private captures may be lost. The screens compiled in a Debug simulator build, but the full validation run still had one separate keyframe-selection test fail at two expectations, and there is no recorded targeted UI or physical-device check for Support. This is not a TestFlight or App Store release.
2026-11-06Making Clipify easier to understand the first timeI added a welcome before permission setup, plus short first-use tips for capture, Journal, and review so someone can understand what each step does before learning by trial and error. Returning or partly configured users are not forced through the welcome again, and Settings now includes a Getting Started guide that can replay the tips. I moved the capture tip below the live capture status so guidance does not cover current feedback, and its dismissal persists after relaunch. CI reported 537 tests in total: 536 passed and one physical-device Vision integration was skipped on the simulator. The recorded CI did not execute the new UI journeys, and there is no physical-device, VoiceOver, TestFlight, or App Store validation for this change.
2026-11-07 to 2026-11-08Keeping Journal navigation and exported stories readableI found that Journal's date sheet could hide Today at larger text sizes, a selected calendar day could jump to an older date near morning rollover, and an exported diary cover could intermittently omit its planned headline. I made the date sheet expand and scroll for larger text, preserved the explicitly selected day in its page range, and resolved the cover headline before the export animation. The full simulator test runs passed with one physical-device-only Vision check skipped, and focused UI tests covered default and accessibility XXXL date controls. The export change used synthetic frames, and I did not test physical devices, iPad or landscape layouts, populated media, or mixed-media exports. This is source code, not a TestFlight or App Store release.
2026-11-08Keeping saved review choices and cancellations honestI fixed two visible review failures that made saved or intentional choices look wrong. Reopening an automatic capture now restores its saved Yes, No, or Not sure answer instead of making a saved No look unanswered. Cancelling the Photos deletion prompt now returns quietly to the unchanged capture viewer instead of showing a misleading permissions error. Simulator checks using the production Gallery UI reopened persisted feedback and exercised two deletion-cancellation and relaunch cycles with temporary synthetic fixtures. After restoring the production root, full test runs reported 40 XCTest cases plus 500 and then 501 Swift Testing cases, with one physical-device Vision check skipped. The scenarios used persisted synthetic captures and photos; I did not test these changes on a physical device, with live capture, with a successful deletion, or across iPad, landscape, and large-text layouts. These are source changes, not a TestFlight or App Store release.
2026-11-16Reviewing an activity with nearby Journal momentsA single capture could have an uncertain activity label even when nearby moments in the same episode made the context clearer. I added a local Review activity action that compares one saved moment with bounded neighboring activity evidence using the on-device Foundation model, then shows the original and proposed labels, uncertainty, and supporting previews before an answer is saved. The comparison rechecks source revisions and never overwrites the moment, regenerates a diary, trains preferences, sends media or location details, or contributes to community learning; the answer is stored separately in a protected local file. Synthetic tests and simulator UI fixtures checked stale data, missing media, cancellation, save failure, large text, and Journal routing. A physical iPhone check exercised one actual Journal comparison and local image loading; a separate isolated device check saved and reopened the protected review store. This is still an explicit experiment, not automatic correction, and it does not prove better activity interpretation, video coverage, or diary quality. It is source code only, not a TestFlight or App Store release.
2026-11-17Reviewing episode structure without rewriting the JournalA saved episode could still have the wrong grouping or activity interpretation, but the existing single-capture comparison did not let me inspect the episode as a whole. I replaced that experiment with a local Review episode draft that opens the selected episode and nearby episodes from the same day. I can move, split, merge, or ungroup moments, edit episode and moment interpretations, compare the original grouping, undo up to ten changes, and optionally ask the on-device model to refine the draft in three stages. The draft checks the exact source revision, becomes read-only if captures change, and stays separate from the live Journal, diaries, learning, analytics, and community feedback. I also bounded original-media lookup for diary updates so it can be cancelled or time out, keeps unavailable entries available for Retry or confirmed removal, and only builds from the resolved sources after an explicit choice. Regression and simulator UI tests were added for drafts, cancellation, stale data, missing media, large text, and partial diary recovery. GitHub CI reached the patch-whitespace gate and failed on a trailing blank line before unit tests ran, so none of the added tests were executed there. I found no physical-device check for this commit. The feature still needs real-day review to show that grouping or interpretations improve, and it is source code only, not a TestFlight or App Store release.
2026-11-20Applying episode refinements inside the JournalThe earlier episode review stayed in a separate draft, so it could help me inspect a proposed grouping but could not change what the Journal showed. I moved refinement into the Journal itself: it can show progress and capture previews in place, then lets me Apply, Discard, or undo a proposed grouping or interpretation, and the accepted Journal view survives relaunch. The original captures, capture-learning answers, and video diary remain unchanged. I also made a stale or overlapping review refuse to apply, retry one structurally invalid on-device model response without replacing the draft, and keep recovery-only capture groups out of the actionable review count. Activity and setting choices now stay in stable positions and can include bounded nearby place categories as unconfirmed suggestions; the lookup does not assert a visit or start continuous location tracking. The source adds focused unit and simulator UI tests for apply, discard, undo, relaunch, large text, failed model output, review recovery, and preview loading, but I found no recorded executed test run or physical-device check for this commit. It is source code only, not a TestFlight or App Store release, and real Journal usefulness and nearby-place accuracy remain unverified.

Looking back

ChestCam helped me explore whether software could suggest moments from signals and patterns. Clipify pushed the idea toward an everyday, user-controlled review experience. Rewriting the app taught me that carrying forward an idea does not always mean carrying forward the original code.

The hardest parts were not always the visible features. A lot of the work was deciding who owned the camera, microphone, saving, learned state, background recovery, and privacy boundaries. The app is still experimental, especially around camera lifecycle, diary behavior, and episode quality on real devices.

Building Clipify also made me realize that I had spent more time studying how a phone detects a moment than how a person chooses one. Someone may press the shutter because of timing, composition, light, or something that is difficult to turn into a score. I requested Photography for senior year because I want to understand that decision better and question what Clipify measures, not just add more detectors.