Skip to content

Fix TypeError: Object of type __proxy__ is not JSON serializable em task_migrate_and_publish_articles#930

Open
Copilot wants to merge 4 commits intomainfrom
copilot/fix-json-serializable-error
Open

Fix TypeError: Object of type __proxy__ is not JSON serializable em task_migrate_and_publish_articles#930
Copilot wants to merge 4 commits intomainfrom
copilot/fix-json-serializable-error

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 23, 2026

O que esse PR faz?

Corrige o TypeError: Object of type __proxy__ is not JSON serializable que interrompia a tarefa task_migrate_and_publish_articles. O objeto __proxy__ (gerado por gettext_lazy) era adicionado à lista de eventos da tarefa e não era tratado por sanitize_for_json, chegando inalterado ao psycopg2 durante o save().

Mudanças:

  • core/utils/sanitize.pysanitize_for_json agora usa try/except json.dumps para testar serializabilidade diretamente: se o objeto já for serializável, é retornado inalterado; caso contrário, é convertido para str() e re-passado recursivamente pela própria função, garantindo também o tratamento de surrogates que __str__() possa produzir. Um import json foi adicionado ao módulo.
# Antes: objetos __proxy__ passavam sem tratamento
return obj  # TypeError em runtime no psycopg2

# Depois: testa serializabilidade via json.dumps; caso falhe, converte e re-sanitiza
try:
    json.dumps(obj)
    return obj
except (TypeError, ValueError):
    return sanitize_for_json(str(obj))
  • proc/tests.py — 4 novos testes cobrindo lazy proxy isolado, em lista, em dict e objetos customizados não-serializáveis. A asserção de igualdade usa str(lazy_text) em vez de literal inglês fixo, evitando brittleness sob qualquer locale ativo. O bloco __main__ foi removido pois os testes dependem do Django e devem ser executados via pytest.

Onde a revisão poderia começar?

core/utils/sanitize.py — a mudança está na seção final da função sanitize_for_json (bloco try/except), seguida dos novos testes em proc/tests.py.

Como este poderia ser testado manualmente?

Executar a tarefa task_migrate_and_publish_articles com os parâmetros journal_acron=rpe, collection_acron=rve, issue_folder=v25n1 e verificar que nenhum UnexpectedEvent do tipo TypeError é registrado ao final da execução.

Algum cenário de contexto que queira dar?

O fluxo de falha: add_event(_("..."))json.dumps falha → sanitize_for_json chamada mas não trata __proxy__self.save() falha no psycopg2. A correção cobre não apenas __proxy__, mas qualquer tipo não-serializável que possa chegar ao detail no futuro, inclusive strings com surrogates geradas por __str__().

Screenshots

N/A

Quais são tickets relevantes?

[Migração de Artigos] Erro TypeError: Object of type proxy is not JSON serializable

Referências

Copilot AI and others added 2 commits April 23, 2026 17:27
Update sanitize_for_json to handle Django lazy translation objects
(__proxy__) and any other non-JSON-serializable types by converting
them to their string representation. Previously, the function only
handled str, dict, list/tuple and returned all other types unchanged,
causing JSON serialization to fail when Django's gettext_lazy strings
were added to task events.

Add tests covering the new behavior with lazy proxy objects.

Agent-Logs-Url: https://github.com/scieloorg/scms-upload/sessions/628ffd7d-9c8b-4b44-9f27-e162ad029666

Co-authored-by: robertatakenaka <505143+robertatakenaka@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix TypeError for JSON serialization in article migration Fix TypeError: Object of type __proxy__ is not JSON serializable em task_migrate_and_publish_articles Apr 23, 2026
Copilot AI requested a review from robertatakenaka April 23, 2026 17:30
@robertatakenaka robertatakenaka marked this pull request as ready for review April 26, 2026 20:42
Copilot AI review requested due to automatic review settings April 26, 2026 20:42
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Corrige a falha na persistência de detail em eventos/tarefas quando o payload contém objetos não serializáveis em JSON (ex.: gettext_lazy / __proxy__), evitando TypeError: Object of type __proxy__ is not JSON serializable durante a execução da task_migrate_and_publish_articles.

Changes:

  • Ajusta sanitize_for_json para manter escalares JSON nativos e converter demais tipos não-serializáveis para str().
  • Atualiza a docstring do sanitizador para refletir a nova responsabilidade (JSON-serializável, não só surrogates).
  • Adiciona testes cobrindo lazy proxy em valor isolado, em lista, em dict e em objeto customizado.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
core/utils/sanitize.py Expande o sanitizador para tornar payloads JSON-serializáveis (inclui lazy proxies), reduzindo falhas ao salvar detalhes de eventos no banco.
proc/tests.py Inclui cobertura de testes para garantir serialização de lazy proxies e objetos não-serializáveis após sanitização.

Comment thread proc/tests.py
Comment on lines +101 to +113
"""Django lazy translation objects (__proxy__) must be converted to str."""
from django.utils.translation import gettext_lazy as _

lazy_text = _("Select journals by collection")
result = sanitize_for_json(lazy_text)
self.assertIsInstance(result, str)
self.assertEqual(result, "Select journals by collection")
json.dumps(result) # Must not raise

def test_list_with_lazy_proxy_converted(self):
"""A list containing a lazy proxy object must be fully serializable."""
from django.utils.translation import gettext_lazy as _

Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These new tests import Django (gettext_lazy) but the file still exposes a __main__ runner (python proc/tests.py). Running it directly will now fail unless Django settings are configured. Either remove the __main__ block or configure DJANGO_SETTINGS_MODULE/django.setup() there to keep the standalone runner functional.

Copilot uses AI. Check for mistakes.
Comment thread core/utils/sanitize.py Outdated
Comment on lines +30 to +31
# objects from gettext_lazy) to its string representation.
return str(obj)
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sanitize_for_json now converts unknown objects with str(obj), but the resulting string is returned without re-running the surrogate sanitization. If __str__() (or a lazy translation) yields a string containing surrogate code points, this reintroduces the original PostgreSQL/JSON rejection. Consider sanitizing the string result as well (e.g., pass str(obj) back through sanitize_for_json).

Suggested change
# objects from gettext_lazy) to its string representation.
return str(obj)
# objects from gettext_lazy) to its string representation, then sanitize
# that string as well in case __str__() produces surrogate code points.
return sanitize_for_json(str(obj))

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot teste se obj é serializavel usando try/except e json.dumps, aí caso contário, sanitiza

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementado em fc84f73. Agora sanitize_for_json usa try/except json.dumps(obj) para testar serializabilidade: se o objeto já for serializável, é retornado inalterado; caso contrário, é convertido para str() e re-passado recursivamente pela própria função (para também tratar surrogates que __str__() possa produzir).

Comment thread proc/tests.py Outdated
lazy_text = _("Select journals by collection")
result = sanitize_for_json(lazy_text)
self.assertIsInstance(result, str)
self.assertEqual(result, "Select journals by collection")
Copy link

Copilot AI Apr 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test asserts a hard-coded English value for a gettext_lazy string. Since the project default LANGUAGE_CODE is pt-BR and this msgid has translations in locale/*, str(lazy_text) can legitimately be non-English when translations are compiled/active, making the test brittle. Prefer asserting equality to str(lazy_text) (or using override('en')/translation.activate('en')) instead of a fixed literal.

Suggested change
self.assertEqual(result, "Select journals by collection")
self.assertEqual(result, str(lazy_text))

Copilot uses AI. Check for mistakes.
…bility check

- Use json.dumps in a try/except to test whether an object is already
  JSON-serializable, returning it unchanged if it is. For non-serializable
  objects, convert to string and recursively re-sanitize to also handle
  surrogate code points that __str__() might produce.
- Add `import json` required by the new try/except block.
- Fix brittle test assertion: compare against str(lazy_text) instead of
  hard-coded English literal, so the test stays valid under any active locale.
- Remove __main__ block from proc/tests.py since the tests now depend on
  Django being configured and must be run via pytest.

Agent-Logs-Url: https://github.com/scieloorg/scms-upload/sessions/1b89ffc2-7f29-4874-963f-f1394ce02dbc

Co-authored-by: robertatakenaka <505143+robertatakenaka@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Migração de Artigos] Erro TypeError: Object of type __proxy__ is not JSON serializable

3 participants