Extracts per-message content into a typed `blocks` list (text, code, thinking, tool_use, tool_result, image_placeholder, file_placeholder, unknown) and renders them at exporter write time. Voice transcripts, Custom Instructions, and image references now appear in exports instead of being silently dropped. Foundation: - src/blocks.py: pure block constructors, _safe_fence (fence-corruption defense, verified live in Joplin), _blockquote_prefix, render - src/loss_report.py: per-run tally surfaced as INFO summary at end of export so silently-dropped data becomes visible Providers: - ChatGPT: dispatch on content_type produces typed blocks; voice shapes (audio_transcription, audio_asset_pointer, real_time_user_audio_video_ asset_pointer) locked from live DevTools capture; Custom Instructions bug fix (parts-vs-direct-fields); role filter lifted; hidden-context marker driven by is_visually_hidden_from_conversation flag - Claude: defensive dispatch for text/thinking/tool_use/tool_result/image with recursive nested-block flattening; untested against real rich- content data — fix-forward in v0.4.1 Exporter: - Markdown renders from blocks at write time via render_blocks_to_markdown; backward-compat fallback to content for any pre-v0.4.0 cached data Tests: - 27 new tests across providers, exporters, CLI; fixtures rebuilt with real-shape ChatGPT voice + Custom Instructions cases - 181/181 pass Behavior changes (intentional): - JSON output omits content; consumers should read blocks - Per-conversation message counts increase (Custom Instructions, image- only, tool-only messages now appear) - Existing exports not auto-re-rendered; users wanting fresh output run cache --clear then export Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
4.4 KiB
4.4 KiB
Changelog
All notable changes to this project will be documented here. Format follows Keep a Changelog.
[0.4.0] - Unreleased
Added
- Rich content support: messages now carry an ordered
blockslist (text, code, thinking, tool_use, tool_result, citation, image_placeholder, file_placeholder, unknown) - ChatGPT voice mode:
audio_transcriptionparts render as text blocks;audio_asset_pointerandreal_time_user_audio_video_asset_pointerrender as📎 File attachedplaceholders with size and duration metadata - ChatGPT Custom Instructions:
user_editable_contextandmodel_editable_contextmessages now appear in exports (were silently dropped — pre-existing bug fixed); rendered with a> ℹ️ Hidden contextmarker driven by theis_visually_hidden_from_conversationflag - Image placeholders for
image_asset_pointerparts (uploads + DALL-E) insidemultimodal_textand at message level - Defensive Claude block extraction:
text,thinking,tool_use,tool_result(including nested-block flattening),imageblocks (untested against real data; will fix-forward in v0.4.1 if real shapes diverge) LossReportsummary table emitted at end of everyexportrun, breaking downunknown blocksandextraction failuresby raw type so silently-dropped data becomes visible_safe_fencehelper picks a fence longer than any backtick run in extracted content, preventing embedded triple-backticks from corrupting downstream rendering (verified live in Joplin during planning)unknownblocks render as> ⚠️ Unsupported contentwith the raw type, observed top-level keys, and reason — so future API additions are visible rather than silent
Changed
- ChatGPT role filter (previously dropped
toolandsystemmessages) is lifted: all roles now route through normal extraction; truly empty messages skip via the existing empty-content guard - Markdown rendering moves from provider-time to exporter-write-time. Providers produce blocks; exporters call
render_blocks_to_markdownat write time. This unblocks future Obsidian/HTML exporters BaseProvider.normalize_conversationsignature now accepts an optionalLossReportparameter (breaking change for any future custom subclass; FileProvider hasn't shipped yet)o1/o3reasoning subparts insidetextcontent_type messages remain rendered as plain text (defensive; reclassification tothinkingblock deferred until live shape is captured)
Fixed
user_editable_context/model_editable_contextextraction (parts-vs-direct-fields mismatch) — Custom Instructions are no longer silently dropped from every conversation
Migration
- Existing exports are not re-rendered automatically. To pick up v0.4.0 rendering for previously exported conversations:
python -m src.main cache --clear python -m src.main export --provider all - JSON exports: messages now contain
blocks(typed structured content) and may omit the legacycontentfield. External consumers reading JSON should preferblocks. - Per-conversation message counts may increase: previously-dropped Custom Instructions, image-only user turns, and tool-only assistant turns now appear.
Out of scope (deferred to v0.5.0+)
- Binary downloads of images and audio assets (placeholders show metadata only;
content not preserved in this export) - Joplin resource upload for embedded media
- Filename resolution for
file-XYZ/sediment://references - Speculative ChatGPT types (
tether_browsing_display,tether_quote) and DALL-E assistant images — fall through tounknownblocks if encountered
[0.2.0] - Unreleased
Added
- Joplin import automation:
joplincommand syncs exported Markdown files to Joplin as notes - Notebooks created automatically per provider+project (
ChatGPT - My Project, etc.) - Re-running is safe: notes are updated, not duplicated (Joplin note ID stored in manifest)
JOPLIN_API_TOKEN,JOPLIN_API_URL,JOPLIN_REQUEST_TIMEOUTconfig variables- Configurable request timeout with clear error messages and actionable hints on timeout
--projectfilter onexportandlistcommands (case-insensitive substring ornone)- ChatGPT Projects support via
CHATGPT_PROJECT_IDSenv var
[0.1.0] - Unreleased
Added
- Initial implementation: ChatGPT and Claude export via internal web APIs
- Markdown and JSON exporters
- Local cache/manifest for incremental sync
- CLI with export, list, cache, doctor, and auth commands