增加Teleram官方的JSON格式导出历史记录适配

[ModerRAS]

[github-actions]

AI Implementation Plan

Issue #81: 增加Telegram官方的JSON格式导出历史记录适配

1. Problem Summary

The issue requests adding support for Telegram's official JSON format chat history export. Currently, the bot has a basic ChatExport model and ChatImportService that handles some Telegram export scenarios, but there are likely gaps in:

• Missing message type handling (service messages, voice notes, video notes, stickers, polls, etc.)
• Missing export fields (forwarded messages, reactions, edit history, etc.)
• Incomplete text processing (only Text_Entities handled, not simple Text array)
• Edge cases for different chat types (personal chat, private group, supergroup, channel)

2. Modules Affected

File	Purpose	Change Type
TelegramSearchBot/Model/ChatExport/ChatExport.cs	Data models for Telegram export	Enhance
TelegramSearchBot/Service/Manage/ChatImportService.cs	Import logic	Enhance
TelegramSearchBot/Controller/Manage/RefreshController.cs	Command triggers	Possibly add new commands
TelegramSearchBot/Service/Manage/RefreshService.cs	Service orchestration	May add new methods

3. Step-by-Step Implementation Plan

Phase 1: Model Enhancement

1. Add missing message type fields:

   • Forwarded_From / ForwardedMessage - for forwarded messages
   • Reactions - for message reactions
   • Via_Bot - for messages via bot
   • Stub - for message stubs (deleted messages)
   • Video / VideoFileSize / DurationSeconds - for video messages
   • Voice / VoiceFileSize - for voice messages
   • VideoNote - for video notes
   • Sticker - for stickers
   • Poll - for poll messages
   • Dice - for dice/emoji reactions
   • Game - for games
   • Invoice - for invoices
   • PaymentInfo - for payment info
   • Location - for location messages
   • Contact - for contact sharing

2. Add chat-level export metadata:

   • MessagesCount - total message count
   • MediaCount - media file count

3. Add missing message types:

   • service - service messages (user joined, left, etc.)
   • encrypted - secret chat messages
   • game_score - game score updates

Phase 2: Import Service Enhancement

4. Update GetMessageText method:

   • Handle simple Text array (currently only handles Text_Entities)
   • Combine both Text and Text_Entities for backward compatibility

5. Add handling for new media types:

   • Video messages (Video field)
   • Voice notes (Voice field)
   • Video notes (VideoNote field)
   • Stickers (save as .webp)

6. Add handling for message types:

   • Forwarded messages - preserve forward info
   • Service messages - create appropriate content
   • Reactions - store reaction data if needed

7. Improve import robustness:

   • Add better error handling for missing files
   • Add progress logging for large imports
   • Add batch processing for performance

Phase 3: Command Enhancement (Optional)

8. Add new import commands:
   • Selective import (specific chats/date ranges)
   • Re-import with update capability
   • Import status/progress query

4. Testing Strategy

1. Unit Tests:

   • Test GetMessageText with various entity types
   • Test model deserialization with sample JSON files
   • Test edge cases (null values, missing fields)

2. Integration Tests:

   • Create sample Telegram export JSON files
   • Test import with various chat types
   • Test media file handling

3. Manual Testing:

   • Export actual Telegram chat using official export feature
   • Verify all message types are correctly imported
   • Verify media files are correctly copied

5. Backward Compatibility

• Maintain support for existing import format
• Use nullable types for new optional fields
• Gracefully handle missing fields in older exports

6. References

• Telegram Desktop Export: https://github.com/telegramdesktop/tdesktop/blob/master/Telegram/SourceFiles/export/data/export_data_types.cpp
• Current implementation: Model/ChatExport/ChatExport.cs, Service/Manage/ChatImportService.cs

[ModerRAS]

中文方案

需求摘要

添加对Telegram官方JSON格式聊天历史导出的支持。

问题分析

目前机器人有基本的 ChatExport 模型和 ChatImportService，但在以下方面存在差距：

• 缺少消息类型处理（服务消息、语音消息、视频消息、贴纸、投票等）
• 缺少导出字段（转发消息、反应、编辑历史等）
• 文本处理不完整（仅处理 Text_Entities，未处理简单 Text 数组）
• 不同聊天类型的边缘情况（个人聊天、私有群组、超级群组、频道）

受影响模块

文件	用途	变更类型
Model/ChatExport/ChatExport.cs	Telegram导出的数据模型	增强
Service/Manage/ChatImportService.cs	导入逻辑	增强
Controller/Manage/RefreshController.cs	命令触发器	可能添加新命令

实施计划

第一阶段：模型增强

添加缺失的消息类型字段:

• Forwarded_From / ForwardedMessage - 转发消息
• Reactions - 消息反应
• Via_Bot - 通过Bot发送的消息
• Stub - 消息存根（已删除消息）
• Video / VideoFileSize / DurationSeconds - 视频消息
• Voice / VoiceFileSize - 语音消息
• VideoNote - 视频笔记
• Sticker - 贴纸
• Poll - 投票消息
• Dice - 骰子/表情反应
• Location - 位置消息
• Contact - 联系人分享

添加聊天级别导出元数据:

• MessagesCount - 总消息数
• MediaCount - 媒体文件数

第二阶段：导入服务增强

更新 GetMessageText 方法:

• 处理简单的 Text 数组（当前仅处理 Text_Entities）
• 为向后兼容性组合 Text 和 Text_Entities

添加新媒体类型处理:

• 视频消息（Video 字段）
• 语音消息（Voice 字段）
• 视频笔记（VideoNote 字段）
• 贴纸（保存为 .webp）

改进导入健壮性:

• 更好的缺失文件错误处理
• 大型导入的进度日志
• 性能批处理

向后兼容性

• 保持对现有导入格式的支持
• 使用可空类型处理新的可选字段
• 优雅处理旧导出中的缺失字段

参考

• Telegram Desktop导出源码: https://github.com/telegramdesktop/tdesktop/blob/master/Telegram/SourceFiles/export/data/export_data_types.cpp
• 当前实现: Model/ChatExport/ChatExport.cs, Service/Manage/ChatImportService.cs

---

基于英文方案翻译整理