Catch ResourceGoneException('Issues are disabled for this repo') and skip collecting messages for issues when they arent available

[cdolfi]

Example repo:
https://github.com/debezium/debezium

Exception | ResourceGoneException('Issues are disabled for this repo')

Stack Trace

Traceback (most recent call last):
  File "/augur/.venv/lib/python3.11/site-packages/celery/app/trace.py", line 453, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/augur/.venv/lib/python3.11/site-packages/celery/app/trace.py", line 736, in __protected_call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/augur/augur/tasks/github/messages.py", line 50, in collect_github_messages
    process_large_issue_and_pr_message_collection(repo_id, repo_git, logger, manifest.key_auth, task_name, augur_db, core_data_last_collected)
  File "/augur/augur/tasks/github/messages.py", line 121, in process_large_issue_and_pr_message_collection
    messages = list(github_data_access.paginate_resource(comment_url))
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/augur/augur/tasks/github/util/github_data_access.py", line 64, in paginate_resource
    response = self.make_request_with_retries(url)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/augur/augur/tasks/github/util/github_data_access.py", line 163, in make_request_with_retries
    return self.__make_request_with_retries(url, method, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/augur/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 330, in wrapped_f
    return self(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/augur/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 467, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/augur/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 368, in iter
    result = action(retry_state)
             ^^^^^^^^^^^^^^^^^^^
  File "/augur/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 390, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
                                     ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/augur/.venv/lib/python3.11/site-packages/tenacity/__init__.py", line 470, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/augur/augur/tasks/github/util/github_data_access.py", line 185, in __make_request_with_retries
    result = self.make_request(url, method, timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/augur/augur/tasks/github/util/github_data_access.py", line 141, in make_request
    raise ResourceGoneException(response_msg)
augur.tasks.github.util.github_data_access.ResourceGoneException: Issues are disabled for this repo

[MoralCode]

What's the issue here? That repo does have issues disabled as the error says. Is this impacting other things beyond issue message collection?

[cdolfi]

I don't think so but because it's marked as error core can't complete

[iGufrankhan]

i think
The repository has issues disabled, GitHub returns HTTP 410, and ResourceGoneException is not handled inside process_large_issue_and_pr_message_collection().

We are only catching UrlNotFoundException, but not ResourceGoneException, which is raised when GitHub returns 410 because issues are disabled.

i think this part is in question

for comment_url in comment_urls:
try:
messages = list(github_data_access.paginate_resource(comment_url))
all_data += messages
except UrlNotFoundException:
logger.info(f"{task_name}: PR or issue comment url of {comment_url} returned 404. Skipping.")
skipped_urls += 1

[MoralCode]

We are only catching UrlNotFoundException, but not ResourceGoneException, which is raised when GitHub returns 410 because issues are disabled.

Thats correct. it seems like skipping collection for issues that dont exist makes more sense than raising an error and stopping collection