D>
04:37:35 WARNING pupa: validation of Membership 61f014ee-0220-11f0-8290-deb288e15ee5 failed: 2 validation errors:
Value '--' for field '' does not match regular expression '\A1 \d{3} \d{3}-\d{4}(?: x\d+)?\Z'
Value 'Telephone: --' for field '' does not match regular expression '\A1 \d{3} \d{3}-\d{4}(?: x\d+)?\Z'
|
ca
|
|
2025-03-16 04:37:35
|
Value 'Telephone: --' for field '' does not match regular expression '\A1 \d{3} \d{3}-\d{4}(?: x\d+)?\Z'
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 175, in validate
validator.validate(self.as_dict(), schema)
File "/app/.heroku/python/lib/python3.9/site-packages/validictory/validator.py", line 616, in validate
raise MultipleValidationError(self._errors)
validictory.validator.MultipleValidationError: 2 validation errors:
Value '--' for field '' does not match regular expression '\A1 \d{3} \d{3}-\d{4}(?: x\d+)?\Z'
Value 'Telephone: --' for field '' does not match regular expression '\A1 \d{3} \d{3}-\d{4}(?: x\d+)?\Z'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 104, in do_scrape
self.save_object(obj)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 93, in save_object
self.save_object(obj)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 89, in save_object
raise ve
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 85, in save_object
obj.validate()
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 177, in validate
raise ScrapeValueError('validation of {} {} failed: {}'.format(
pupa.exceptions.ScrapeValueError: validation of Membership 61f014ee-0220-11f0-8290-deb288e15ee5 failed: 2 validation errors:
Value '--' for field '' does not match regular expression '\A1 \d{3} \d{3}-\d{4}(?: x\d+)?\Z'
Value 'Telephone: --' for field '' does not match regular expression '\A1 \d{3} \d{3}-\d{4}(?: x\d+)?\Z'
|
|
D>
|
ca_ab
|
|
2025-03-16 04:25:16
|
obj2 sources: []
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 307, in do_handle
report['import'] = self.do_import(juris, args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 209, in do_import
report.update(post_importer.import_directory(datadir))
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 190, in import_directory
return self.import_data(json_stream())
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 227, in import_data
obj_id, what = self.import_item(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 265, in import_item
raise DuplicateItemError(data, obj, related.get('sources', []))
pupa.exceptions.DuplicateItemError: attempt to import data that would conflict with data already in the import: {'label': 'Bow River', 'role': 'MLA', 'organization_id': 'ocd-organization/745857b7-9ee6-4b87-b190-b3196463d434', 'division_id': 'ocd-division/country:ca/province:ab/ed:bow_river-2015', 'start_date': '', 'end_date': '', 'extras': {}} (already imported as MLA - Bow River - Legislative Assembly of Alberta)
obj1 sources: []
obj2 sources: []
|
D>
04:44:06 WARNING pupa: validation of CanadianPerson 4b4ab8f6-0221-11f0-8290-deb288e15ee5 failed: 1 validation errors:
Value 'Information site' for field '<obj>.name' does not match regular expression 'regex.Regex('\\A(?!(?:Chair|Commissioner|Conseiller|Councillor|Deputy|Dr|M|Maire|Mayor|Miss|Mme|Mr|Mrs|Ms|Regional|Warden)\\b)(?:(?:(?:\\p{Lu}\\.)+|\\p{Lu}+|(?:Jr|Rev|Sr|St)\\.|da|de|den|der|la|van|von|[("](?:\\p{Lu}+|\\p{Lu}\\p{Ll}*(?:-\\p{Lu}\\p{Ll}*)*)[)"]|(?:D\'|d\'|De|de|Des|Di|Du|L\'|La|Le|Mac|Mc|O\'|San|St\\.|Van|Vander?|van|vanden)?\\p{Lu}\\p{Ll}+|\\p{Lu}\\p{Ll}+Anne?|Marie\\p{Lu}\\p{Ll}+|Á\'a:líya|A\'aliya|Ch\'ng|Prud\'homme|Qwulti\'stunaat|Ya\'ara|D!ONNE|ChiefCalf|IsaBelle)(?:\'|-| - | ))+(?:(?:\\p{Lu}\\.)+|\\p{Lu}+|(?:Jr|Rev|Sr|St)\\.|da|de|den|der|la|van|von|[("](?:\\p{Lu}+|\\p{Lu}\\p{Ll}*(?:-\\p{Lu}\\p{Ll}*)*)[)"]|(?:D\'|d\'|De|de|Des|Di|Du|L\'|La|Le|Mac|Mc|O\'|San|St\\.|Van|Vander?|van|vanden)?\\p{Lu}\\p{Ll}+|\\p{Lu}\\p{Ll}+Anne?|Marie\\p{Lu}\\p{Ll}+|Á\'a:líya|A\'aliya|Ch\'ng|Prud\'homme|Qwulti\'stunaat|Ya\'ara|D!ONNE|ChiefCalf|IsaBelle)\\Z', flags=regex.V0)'
|
ca_ab_calgary
|
|
2025-03-16 04:44:06
|
Value 'Information site' for field '<obj>.name' does not match regular expression 'regex.Regex('\\A(?!(?:Chair|Commissioner|…
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 175, in validate
validator.validate(self.as_dict(), schema)
File "/app/.heroku/python/lib/python3.9/site-packages/validictory/validator.py", line 616, in validate
raise MultipleValidationError(self._errors)
validictory.validator.MultipleValidationError: 1 validation errors:
Value 'Information site' for field '<obj>.name' does not match regular expression 'regex.Regex('\\A(?!(?:Chair|Commissioner|Conseiller|Councillor|Deputy|Dr|M|Maire|Mayor|Miss|Mme|Mr|Mrs|Ms|Regional|Warden)\\b)(?:(?:(?:\\p{Lu}\\.)+|\\p{Lu}+|(?:Jr|Rev|Sr|St)\\.|da|de|den|der|la|van|von|[("](?:\\p{Lu}+|\\p{Lu}\\p{Ll}*(?:-\\p{Lu}\\p{Ll}*)*)[)"]|(?:D\'|d\'|De|de|Des|Di|Du|L\'|La|Le|Mac|Mc|O\'|San|St\\.|Van|Vander?|van|vanden)?\\p{Lu}\\p{Ll}+|\\p{Lu}\\p{Ll}+Anne?|Marie\\p{Lu}\\p{Ll}+|Á\'a:líya|A\'aliya|Ch\'ng|Prud\'homme|Qwulti\'stunaat|Ya\'ara|D!ONNE|ChiefCalf|IsaBelle)(?:\'|-| - | ))+(?:(?:\\p{Lu}\\.)+|\\p{Lu}+|(?:Jr|Rev|Sr|St)\\.|da|de|den|der|la|van|von|[("](?:\\p{Lu}+|\\p{Lu}\\p{Ll}*(?:-\\p{Lu}\\p{Ll}*)*)[)"]|(?:D\'|d\'|De|de|Des|Di|Du|L\'|La|Le|Mac|Mc|O\'|San|St\\.|Van|Vander?|van|vanden)?\\p{Lu}\\p{Ll}+|\\p{Lu}\\p{Ll}+Anne?|Marie\\p{Lu}\\p{Ll}+|Á\'a:líya|A\'aliya|Ch\'ng|Prud\'homme|Qwulti\'stunaat|Ya\'ara|D!ONNE|ChiefCalf|IsaBelle)\\Z', flags=regex.V0)'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 104, in do_scrape
self.save_object(obj)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 89, in save_object
raise ve
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 85, in save_object
obj.validate()
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 177, in validate
raise ScrapeValueError('validation of {} {} failed: {}'.format(
pupa.exceptions.ScrapeValueError: validation of CanadianPerson 4b4ab8f6-0221-11f0-8290-deb288e15ee5 failed: 1 validation errors:
Value 'Information site' for field '<obj>.name' does not match regular expression 'regex.Regex('\\A(?!(?:Chair|Commissioner|Conseiller|Councillor|Deputy|Dr|M|Maire|Mayor|Miss|Mme|Mr|Mrs|Ms|Regional|Warden)\\b)(?:(?:(?:\\p{Lu}\\.)+|\\p{Lu}+|(?:Jr|Rev|Sr|St)\\.|da|de|den|der|la|van|von|[("](?:\\p{Lu}+|\\p{Lu}\\p{Ll}*(?:-\\p{Lu}\\p{Ll}*)*)[)"]|(?:D\'|d\'|De|de|Des|Di|Du|L\'|La|Le|Mac|Mc|O\'|San|St\\.|Van|Vander?|van|vanden)?\\p{Lu}\\p{Ll}+|\\p{Lu}\\p{Ll}+Anne?|Marie\\p{Lu}\\p{Ll}+|Á\'a:líya|A\'aliya|Ch\'ng|Prud\'homme|Qwulti\'stunaat|Ya\'ara|D!ONNE|ChiefCalf|IsaBelle)(?:\'|-| - | ))+(?:(?:\\p{Lu}\\.)+|\\p{Lu}+|(?:Jr|Rev|Sr|St)\\.|da|de|den|der|la|van|von|[("](?:\\p{Lu}+|\\p{Lu}\\p{Ll}*(?:-\\p{Lu}\\p{Ll}*)*)[)"]|(?:D\'|d\'|De|de|Des|Di|Du|L\'|La|Le|Mac|Mc|O\'|San|St\\.|Van|Vander?|van|vanden)?\\p{Lu}\\p{Ll}+|\\p{Lu}\\p{Ll}+Anne?|Marie\\p{Lu}\\p{Ll}+|Á\'a:líya|A\'aliya|Ch\'ng|Prud\'homme|Qwulti\'stunaat|Ya\'ara|D!ONNE|ChiefCalf|IsaBelle)\\Z', flags=regex.V0)'
|
|
D>
|
ca_ab_edmonton
|
|
2025-03-16 04:22:12
|
pupa.exceptions.UnresolvedIdError: cannot resolve pseudo id to Post: ~{"label": "Nakota Isga", "organization__classification…
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 307, in do_handle
report['import'] = self.do_import(juris, args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 211, in do_import
report.update(membership_importer.import_directory(datadir))
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 190, in import_directory
return self.import_data(json_stream())
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 227, in import_data
obj_id, what = self.import_item(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 247, in import_item
data = self.prepare_for_db(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/memberships.py", line 50, in prepare_for_db
data['post_id'] = self.post_importer.resolve_json_id(data['post_id'])
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 165, in resolve_json_id
raise UnresolvedIdError(errmsg)
pupa.exceptions.UnresolvedIdError: cannot resolve pseudo id to Post: ~{"label": "Nakota Isga", "organization__classification": "legislature", "role": "Councillor"}
|
|
C
|
ca_ab_grande_prairie
|
2025-03-16 04:20:09
|
2025-03-16 04:20:09
|
|
|
D>
|
ca_ab_grande_prairie_county_no_1
|
|
2025-03-16 04:22:15
|
IndexError: list index out of range
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/ca_ab_grande_prairie_county_no_1/people.py", line 17, in scrape
name = councillor.xpath('.//div[@class="lb-imageBox_header {headColor}"]')[0].text_content()
IndexError: list index out of range
|
|
D>
|
ca_ab_lethbridge
|
|
2025-03-16 04:40:15
|
IndexError: list index out of range
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/ca_ab_lethbridge/people.py", line 38, in scrape
yield self.scrape_mayor()
File "/app/scrapers/ca_ab_lethbridge/people.py", line 12, in scrape_mayor
name = " ".join([paragraph[0], paragraph[1]])
IndexError: list index out of range
|
|
C
|
ca_ab_strathcona_county
|
2025-03-16 04:46:55
|
2025-03-16 04:46:55
|
|
|
C
|
ca_ab_wood_buffalo
|
2025-03-16 04:20:57
|
2025-03-16 04:20:57
|
|
|
D>
|
ca_bc
|
|
2025-03-16 04:39:43
|
obj2 sources: []
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 307, in do_handle
report['import'] = self.do_import(juris, args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 209, in do_import
report.update(post_importer.import_directory(datadir))
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 190, in import_directory
return self.import_data(json_stream())
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 227, in import_data
obj_id, what = self.import_item(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 265, in import_item
raise DuplicateItemError(data, obj, related.get('sources', []))
pupa.exceptions.DuplicateItemError: attempt to import data that would conflict with data already in the import: {'label': 'Delta', 'role': 'MLA', 'organization_id': 'ocd-organization/5a5b1985-efbc-45e3-9cb6-e18b08c50175', 'division_id': 'ocd-division/country:ca/province:bc/ed:delta-2015', 'start_date': '', 'end_date': '', 'extras': {}} (already imported as MLA - Delta - Legislative Assembly of British Columbia)
obj1 sources: []
obj2 sources: []
|
|
C
|
ca_bc_abbotsford
|
2025-03-16 04:22:18
|
2025-03-16 04:22:19
|
|
|
C
|
ca_bc_burnaby
|
2025-03-16 04:37:57
|
2025-03-16 04:37:57
|
|
|
C
|
ca_bc_coquitlam
|
2025-03-16 04:21:08
|
2025-03-16 04:21:08
|
|
|
C
|
ca_bc_kelowna
|
2025-03-16 04:18:20
|
2025-03-16 04:18:20
|
|
|
C
|
ca_bc_langley
|
2025-03-16 04:40:27
|
2025-03-16 04:40:27
|
|
D>
04:38:11 WARNING scrapelib: sleeping for 10 seconds before retry
04:38:21 WARNING scrapelib: sleeping for 20 seconds before retry
04:38:41 WARNING scrapelib: sleeping for 40 seconds before retry
|
ca_bc_langley_city
|
|
2025-03-16 04:39:22
|
scrapelib.HTTPError: 403 while retrieving https://www.langleycity.ca/cityhall/city-council/council-members
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/ca_bc_langley_city/people.py", line 11, in scrape
page = self.lxmlize(COUNCIL_PAGE)
File "/app/scrapers/utils.py", line 217, in lxmlize
response = self.get(url, cookies=cookies, verify=verify)
File "/app/scrapers/utils.py", line 198, in get
return super().get(*args, verify=kwargs.pop("verify", SSL_VERIFY), **kwargs)
File "/app/.heroku/python/lib/python3.9/site-packages/requests/sessions.py", line 602, in get
return self.request("GET", url, **kwargs)
File "/app/.heroku/python/lib/python3.9/site-packages/scrapelib/__init__.py", line 602, in request
raise HTTPError(resp)
scrapelib.HTTPError: 403 while retrieving https://www.langleycity.ca/cityhall/city-council/council-members
|
|
C
|
ca_bc_new_westminster
|
2025-03-16 04:40:13
|
2025-03-16 04:40:13
|
|
|
C
|
ca_bc_richmond
|
2025-03-16 04:38:08
|
2025-03-16 04:38:08
|
|
|
C
|
ca_bc_saanich
|
2025-03-16 04:16:17
|
2025-03-16 04:16:17
|
|
|
C
|
ca_bc_surrey
|
2025-03-16 04:45:41
|
2025-03-16 04:45:41
|
|
|
C
|
ca_bc_vancouver
|
2025-03-16 04:18:16
|
2025-03-16 04:18:16
|
|
|
C
|
ca_bc_victoria
|
2025-03-16 04:14:36
|
2025-03-16 04:14:36
|
|
|
D>
|
ca_mb
|
|
2025-03-16 04:43:47
|
obj2 sources: []
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 307, in do_handle
report['import'] = self.do_import(juris, args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 209, in do_import
report.update(post_importer.import_directory(datadir))
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 190, in import_directory
return self.import_data(json_stream())
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 227, in import_data
obj_id, what = self.import_item(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 265, in import_item
raise DuplicateItemError(data, obj, related.get('sources', []))
pupa.exceptions.DuplicateItemError: attempt to import data that would conflict with data already in the import: {'label': 'Selkirk', 'role': 'MLA', 'organization_id': 'ocd-organization/6a5b8ea5-75b8-4dbd-8083-328c36c4a4ce', 'division_id': 'ocd-division/country:ca/province:mb/ed:selkirk-2018', 'start_date': '', 'end_date': '', 'extras': {}} (already imported as MLA - Selkirk - Legislative Assembly of Manitoba)
obj1 sources: []
obj2 sources: []
|
|
C
|
ca_mb_winnipeg
|
2025-03-16 04:43:52
|
2025-03-16 04:43:52
|
|
|
D>
|
ca_nb
|
|
2025-03-16 04:45:29
|
IndexError: list index out of range
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/ca_nb/people.py", line 16, in scrape
address = node.xpath('//td[contains(text(),"Address")]/parent::tr//td[2]')[0]
IndexError: list index out of range
|
|
C
|
ca_nb_fredericton
|
2025-03-16 04:12:31
|
2025-03-16 04:12:31
|
|
|
C
|
ca_nb_moncton
|
2025-03-16 04:40:30
|
2025-03-16 04:40:30
|
|
|
C
|
ca_nb_saint_john
|
2025-03-16 04:20:32
|
2025-03-16 04:20:32
|
|
|
D>
|
ca_nl
|
|
2025-03-16 04:08:43
|
obj2 sources: []
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 307, in do_handle
report['import'] = self.do_import(juris, args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 209, in do_import
report.update(post_importer.import_directory(datadir))
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 190, in import_directory
return self.import_data(json_stream())
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 227, in import_data
obj_id, what = self.import_item(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 265, in import_item
raise DuplicateItemError(data, obj, related.get('sources', []))
pupa.exceptions.DuplicateItemError: attempt to import data that would conflict with data already in the import: {'label': "St. John's East", 'role': 'MHA', 'organization_id': 'ocd-organization/b4bd506a-4f52-4138-a789-bbdf6878f8a1', 'division_id': 'ocd-division/country:ca/province:nl/ed:st._john~s_east-2004', 'start_date': '', 'end_date': '', 'extras': {}} (already imported as MHA - St. John's East - Newfoundland and Labrador House of Assembly)
obj1 sources: []
obj2 sources: []
|
|
C
|
ca_nl_st_john_s
|
2025-03-16 04:11:06
|
2025-03-16 04:11:06
|
|
|
D>
|
ca_ns
|
|
2025-03-16 04:45:05
|
obj2 sources: []
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 307, in do_handle
report['import'] = self.do_import(juris, args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 209, in do_import
report.update(post_importer.import_directory(datadir))
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 190, in import_directory
return self.import_data(json_stream())
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 227, in import_data
obj_id, what = self.import_item(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 265, in import_item
raise DuplicateItemError(data, obj, related.get('sources', []))
pupa.exceptions.DuplicateItemError: attempt to import data that would conflict with data already in the import: {'label': 'Richmond', 'role': 'MLA', 'organization_id': 'ocd-organization/338a0367-b7d4-4b38-8014-0a807bf06da9', 'division_id': 'ocd-division/country:ca/province:ns/ed:46', 'start_date': '', 'end_date': '', 'extras': {}} (already imported as MLA - Richmond - Nova Scotia House of Assembly)
obj1 sources: []
obj2 sources: []
|
D>
04:11:44 WARNING pupa: validation of CanadianPerson c5a16d34-021c-11f0-8290-deb288e15ee5 failed: 2 validation errors:
Value '' for field '<obj>.name' does not match regular expression 'regex.Regex('\\A(?!(?:Chair|Commissioner|Conseiller|Councillor|Deputy|Dr|M|Maire|Mayor|Miss|Mme|Mr|Mrs|Ms|Regional|Warden)\\b)(?:(?:(?:\\p{Lu}\\.)+|\\p{Lu}+|(?:Jr|Rev|Sr|St)\\.|da|de|den|der|la|van|von|[("](?:\\p{Lu}+|\\p{Lu}\\p{Ll}*(?:-\\p{Lu}\\p{Ll}*)*)[)"]|(?:D\'|d\'|De|de|Des|Di|Du|L\'|La|Le|Mac|Mc|O\'|San|St\\.|Van|Vander?|van|vanden)?\\p{Lu}\\p{Ll}+|\\p{Lu}\\p{Ll}+Anne?|Marie\\p{Lu}\\p{Ll}+|Á\'a:líya|A\'aliya|Ch\'ng|Prud\'homme|Qwulti\'stunaat|Ya\'ara|D!ONNE|ChiefCalf|IsaBelle)(?:\'|-| - | ))+(?:(?:\\p{Lu}\\.)+|\\p{Lu}+|(?:Jr|Rev|Sr|St)\\.|da|de|den|der|la|van|von|[("](?:\\p{Lu}+|\\p{Lu}\\p{Ll}*(?:-\\p{Lu}\\p{Ll}*)*)[)"]|(?:D\'|d\'|De|de|Des|Di|Du|L\'|La|Le|Mac|Mc|O\'|San|St\\.|Van|Vander?|van|vanden)?\\p{Lu}\\p{Ll}+|\\p{Lu}\\p{Ll}+Anne?|Marie\\p{Lu}\\p{Ll}+|Á\'a:líya|A\'aliya|Ch\'ng|Prud\'homme|Qwulti\'stunaat|Ya\'ara|D!ONNE|ChiefCalf|IsaBelle)\\Z', flags=regex.V0)'
Value '' for field '<obj>.name' cannot be blank'
|
ca_ns_cape_breton
|
|
2025-03-16 04:11:44
|
Value '' for field '<obj>.name' cannot be blank'
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 175, in validate
validator.validate(self.as_dict(), schema)
File "/app/.heroku/python/lib/python3.9/site-packages/validictory/validator.py", line 616, in validate
raise MultipleValidationError(self._errors)
validictory.validator.MultipleValidationError: 2 validation errors:
Value '' for field '<obj>.name' does not match regular expression 'regex.Regex('\\A(?!(?:Chair|Commissioner|Conseiller|Councillor|Deputy|Dr|M|Maire|Mayor|Miss|Mme|Mr|Mrs|Ms|Regional|Warden)\\b)(?:(?:(?:\\p{Lu}\\.)+|\\p{Lu}+|(?:Jr|Rev|Sr|St)\\.|da|de|den|der|la|van|von|[("](?:\\p{Lu}+|\\p{Lu}\\p{Ll}*(?:-\\p{Lu}\\p{Ll}*)*)[)"]|(?:D\'|d\'|De|de|Des|Di|Du|L\'|La|Le|Mac|Mc|O\'|San|St\\.|Van|Vander?|van|vanden)?\\p{Lu}\\p{Ll}+|\\p{Lu}\\p{Ll}+Anne?|Marie\\p{Lu}\\p{Ll}+|Á\'a:líya|A\'aliya|Ch\'ng|Prud\'homme|Qwulti\'stunaat|Ya\'ara|D!ONNE|ChiefCalf|IsaBelle)(?:\'|-| - | ))+(?:(?:\\p{Lu}\\.)+|\\p{Lu}+|(?:Jr|Rev|Sr|St)\\.|da|de|den|der|la|van|von|[("](?:\\p{Lu}+|\\p{Lu}\\p{Ll}*(?:-\\p{Lu}\\p{Ll}*)*)[)"]|(?:D\'|d\'|De|de|Des|Di|Du|L\'|La|Le|Mac|Mc|O\'|San|St\\.|Van|Vander?|van|vanden)?\\p{Lu}\\p{Ll}+|\\p{Lu}\\p{Ll}+Anne?|Marie\\p{Lu}\\p{Ll}+|Á\'a:líya|A\'aliya|Ch\'ng|Prud\'homme|Qwulti\'stunaat|Ya\'ara|D!ONNE|ChiefCalf|IsaBelle)\\Z', flags=regex.V0)'
Value '' for field '<obj>.name' cannot be blank'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 104, in do_scrape
self.save_object(obj)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 89, in save_object
raise ve
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 85, in save_object
obj.validate()
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 177, in validate
raise ScrapeValueError('validation of {} {} failed: {}'.format(
pupa.exceptions.ScrapeValueError: validation of CanadianPerson c5a16d34-021c-11f0-8290-deb288e15ee5 failed: 2 validation errors:
Value '' for field '<obj>.name' does not match regular expression 'regex.Regex('\\A(?!(?:Chair|Commissioner|Conseiller|Councillor|Deputy|Dr|M|Maire|Mayor|Miss|Mme|Mr|Mrs|Ms|Regional|Warden)\\b)(?:(?:(?:\\p{Lu}\\.)+|\\p{Lu}+|(?:Jr|Rev|Sr|St)\\.|da|de|den|der|la|van|von|[("](?:\\p{Lu}+|\\p{Lu}\\p{Ll}*(?:-\\p{Lu}\\p{Ll}*)*)[)"]|(?:D\'|d\'|De|de|Des|Di|Du|L\'|La|Le|Mac|Mc|O\'|San|St\\.|Van|Vander?|van|vanden)?\\p{Lu}\\p{Ll}+|\\p{Lu}\\p{Ll}+Anne?|Marie\\p{Lu}\\p{Ll}+|Á\'a:líya|A\'aliya|Ch\'ng|Prud\'homme|Qwulti\'stunaat|Ya\'ara|D!ONNE|ChiefCalf|IsaBelle)(?:\'|-| - | ))+(?:(?:\\p{Lu}\\.)+|\\p{Lu}+|(?:Jr|Rev|Sr|St)\\.|da|de|den|der|la|van|von|[("](?:\\p{Lu}+|\\p{Lu}\\p{Ll}*(?:-\\p{Lu}\\p{Ll}*)*)[)"]|(?:D\'|d\'|De|de|Des|Di|Du|L\'|La|Le|Mac|Mc|O\'|San|St\\.|Van|Vander?|van|vanden)?\\p{Lu}\\p{Ll}+|\\p{Lu}\\p{Ll}+Anne?|Marie\\p{Lu}\\p{Ll}+|Á\'a:líya|A\'aliya|Ch\'ng|Prud\'homme|Qwulti\'stunaat|Ya\'ara|D!ONNE|ChiefCalf|IsaBelle)\\Z', flags=regex.V0)'
Value '' for field '<obj>.name' cannot be blank'
|
|
D>
|
ca_ns_halifax
|
|
2025-03-16 04:09:09
|
pupa.exceptions.UnresolvedIdError: cannot resolve pseudo id to Post: ~{"label": "Hammonds Plains\u2014Upper Hammonds Plains\…
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 307, in do_handle
report['import'] = self.do_import(juris, args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 211, in do_import
report.update(membership_importer.import_directory(datadir))
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 190, in import_directory
return self.import_data(json_stream())
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 227, in import_data
obj_id, what = self.import_item(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 247, in import_item
data = self.prepare_for_db(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/memberships.py", line 50, in prepare_for_db
data['post_id'] = self.post_importer.resolve_json_id(data['post_id'])
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 165, in resolve_json_id
raise UnresolvedIdError(errmsg)
pupa.exceptions.UnresolvedIdError: cannot resolve pseudo id to Post: ~{"label": "Hammonds Plains\u2014Upper Hammonds Plains\u2014Lucasville\u2014Middle and Upper Sackville", "organization__classification": "legislature", "role": "Councillor"}
|
|
D>
|
ca_nt
|
|
2025-03-16 04:11:29
|
obj2 sources: []
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 307, in do_handle
report['import'] = self.do_import(juris, args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 209, in do_import
report.update(post_importer.import_directory(datadir))
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 190, in import_directory
return self.import_data(json_stream())
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 227, in import_data
obj_id, what = self.import_item(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 265, in import_item
raise DuplicateItemError(data, obj, related.get('sources', []))
pupa.exceptions.DuplicateItemError: attempt to import data that would conflict with data already in the import: {'label': 'Northwest Territories', 'role': 'MLA', 'organization_id': 'ocd-organization/6a064834-529d-4851-ab76-2fd5ddb1da19', 'division_id': 'ocd-division/country:ca/territory:nt/ed:northwest_territories-2015', 'start_date': '', 'end_date': '', 'extras': {}} (already imported as MLA - Northwest Territories - Legislative Assembly of the Northwest Territories)
obj1 sources: []
obj2 sources: []
|
|
D>
|
ca_nu
|
2025-03-16 04:21:56
|
2025-03-16 04:21:56
|
DETAIL: Key (division_id)=(ocd-division/country:ca/territory:nu/ed:nunavut) is not present in table "opencivicdata_division…
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.9/site-packages/django/db/backends/base/base.py", line 313, in _commit
return self.connection.commit()
psycopg2.errors.ForeignKeyViolation: insert or update on table "opencivicdata_post" violates foreign key constraint "opencivicdata_post_division_id_82fef8df_fk_opencivic"
DETAIL: Key (division_id)=(ocd-division/country:ca/territory:nu/ed:nunavut) is not present in table "opencivicdata_division".
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/reports/utils.py", line 75, in scrape_people
report.success_at = datetime.datetime.now(datetime.timezone.utc)
File "/app/.heroku/python/lib/python3.9/site-packages/django/db/transaction.py", line 263, in __exit__
connection.commit()
File "/app/.heroku/python/lib/python3.9/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/app/.heroku/python/lib/python3.9/site-packages/django/db/backends/base/base.py", line 337, in commit
self._commit()
File "/app/.heroku/python/lib/python3.9/site-packages/django/db/backends/base/base.py", line 313, in _commit
return self.connection.commit()
File "/app/.heroku/python/lib/python3.9/site-packages/django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/app/.heroku/python/lib/python3.9/site-packages/django/db/backends/base/base.py", line 313, in _commit
return self.connection.commit()
django.db.utils.IntegrityError: insert or update on table "opencivicdata_post" violates foreign key constraint "opencivicdata_post_division_id_82fef8df_fk_opencivic"
DETAIL: Key (division_id)=(ocd-division/country:ca/territory:nu/ed:nunavut) is not present in table "opencivicdata_division".
|
|
D>
|
ca_on
|
|
2025-03-16 04:25:27
|
AssertionError: No members found
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/ca_on/people.py", line 20, in scrape
assert len(members), "No members found"
AssertionError: No members found
|
|
C
|
ca_on_ajax
|
2025-03-16 04:20:52
|
2025-03-16 04:20:52
|
|
|
C
|
ca_on_belleville
|
2025-03-16 04:21:31
|
2025-03-16 04:21:31
|
|
|
C
|
ca_on_brampton
|
2025-03-16 04:14:15
|
2025-03-16 04:14:15
|
|
|
C
|
ca_on_brantford
|
2025-03-16 04:22:30
|
2025-03-16 04:22:30
|
|
|
C
|
ca_on_burlington
|
2025-03-16 04:19:49
|
2025-03-16 04:19:49
|
|
|
C
|
ca_on_caledon
|
2025-03-16 04:22:53
|
2025-03-16 04:22:53
|
|
|
C
|
ca_on_cambridge
|
2025-03-16 04:39:46
|
2025-03-16 04:39:46
|
|
|
C
|
ca_on_chatham_kent
|
2025-03-16 04:08:33
|
2025-03-16 04:08:33
|
|
|
C
|
ca_on_clarington
|
2025-03-16 04:45:45
|
2025-03-16 04:45:45
|
|
|
C
|
ca_on_fort_erie
|
2025-03-16 04:18:12
|
2025-03-16 04:18:12
|
|
|
C
|
ca_on_georgina
|
2025-03-16 04:22:27
|
2025-03-16 04:22:27
|
|
|
C
|
ca_on_greater_sudbury
|
2025-03-16 04:25:30
|
2025-03-16 04:25:30
|
|
|
C
|
ca_on_grimsby
|
2025-03-16 04:01:50
|
2025-03-16 04:01:50
|
|
|
C
|
ca_on_guelph
|
2025-03-16 04:18:23
|
2025-03-16 04:18:23
|
|
|
D>
|
ca_on_haldimand_county
|
|
2025-03-16 04:25:24
|
AssertionError: No councillors found
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/ca_on_haldimand_county/people.py", line 12, in scrape
assert len(councillors), "No councillors found"
AssertionError: No councillors found
|
|
C
|
ca_on_hamilton
|
2025-03-16 04:20:29
|
2025-03-16 04:20:29
|
|
|
C
|
ca_on_huron
|
2025-03-16 04:46:31
|
2025-03-16 04:46:31
|
|
|
D>
|
ca_on_kawartha_lakes
|
|
2025-03-16 04:18:18
|
Exception: No email node in b'<tr>\n<td>\n<p>The Council seat for Ward 5 previously held by Councillor Eric Smeaton is now v…
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/ca_on_kawartha_lakes/people.py", line 30, in scrape
email = self.get_email(info_node)
File "/app/scrapers/utils.py", line 140, in get_email
raise Exception(f"No email node in {etree.tostring(node)}")
Exception: No email node in b'<tr>\n<td>\n<p>The Council seat for Ward 5 previously held by Councillor Eric Smeaton is now vacant. </p>\n<p>Inquiries can be directed to the Mayor, Deputy Mayor or the Executive Assistant to Mayor and Council.</p>\n<p>Telephone: 705-324-9411 extension 1310</p>\n<table class="icrtAccordion" border="0">\n<tbody>\n<tr>\n<td> Ward 5 Boundary</td>\n</tr>\n<tr>\n<td>\n<p><a href="https://www.kawarthalakes.ca/en/municipal-services/resources/Maps/Individual-Ward-Maps-1-8_Ward5.pdf" target="_blank" aria-label="View a detailed map of Ward 5 opens in new window">View a detailed map of Ward 5</a><br/>North - West ½ of Lot 28 Concession 8 Ops; lands lying south of Lot 29 west ½ Concessions 6, 7 and 8 to Sturgeon Lake; follow Lake to south boundary of Ward 3</p>\n<p>East - West ½ of Lot 21 to 28 Concession 8 Ops</p>\n<p>South - Dew Drop Inn Road to Hwy 7B/35 to Kent Street West, jog Lindsay Street North to Kent Street East to west ½ of Lot 21 Concession 8 Ops</p>\n<p>West - East ½ Lot 22 Concession 10 - intersection adjacent to Dew Drop Inn Road.</p>\n<div class="AccordionContent">\n<p><strong><a title="Open file in new window: Individual-Ward-Maps-1-8_Ward5.pdf" href="https://www.kawarthalakes.ca/en/municipal-services/resources/Maps/Individual-Ward-Maps-1-8_Ward5.pdf" target="_blank" aria-label="View a detailed map of Ward 5 opens in new window">View a detailed map of Ward 5</a><br/>North</strong> - West ½ of Lot 28 Concession 8 Ops; lands lying south of Lot 29 west ½ Concessions 6, 7 and 8 to Sturgeon Lake; follow Lake to south boundary of Ward 3</p>\n<p><strong>East</strong> - West ½ of Lot 21 to 28 Concession 8 Ops</p>\n<p><strong>South</strong> - Dew Drop Inn Road to Hwy 7B/35 to Kent Street West, jog Lindsay Street North to Kent Street East to west ½ of Lot 21 Concession 8 Ops</p>\n<p><strong>West - </strong> East ½ Lot 22 Concession 10 - intersection adjacent to Dew Drop Inn Road.</p>\n</div>\n</td>\n</tr>\n</tbody>\n</table>\n</td>\n</tr>\n'
|
|
C
|
ca_on_king
|
2025-03-16 04:10:29
|
2025-03-16 04:10:29
|
|
|
D>
|
ca_on_kingston
|
|
2025-03-16 04:37:37
|
KeyError: 'name'
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/utils.py", line 420, in scrape
if not self.is_valid_row(row):
File "/app/scrapers/utils.py", line 374, in is_valid_row
return row["name"] not in empty
KeyError: 'name'
|
|
C
|
ca_on_kitchener
|
2025-03-16 04:39:27
|
2025-03-16 04:39:27
|
|
|
D>
|
ca_on_lambton
|
|
2025-03-16 04:37:39
|
IndexError: list index out of range
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/ca_on_lambton/people.py", line 15, in scrape
text = councillor.xpath(".//h3/text()")[0]
IndexError: list index out of range
|
|
C
|
ca_on_lasalle
|
2025-03-16 04:39:24
|
2025-03-16 04:39:24
|
|
|
C
|
ca_on_lincoln
|
2025-03-16 04:39:51
|
2025-03-16 04:39:52
|
|
|
C
|
ca_on_london
|
2025-03-16 04:21:28
|
2025-03-16 04:21:28
|
|
|
D>
|
ca_on_markham
|
|
2025-03-16 04:39:54
|
IndexError: list index out of range
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 101, in do_scrape
for iterobj in obj:
File "/app/scrapers/ca_on_markham/people.py", line 87, in scrape_mayor
name = page.xpath(
IndexError: list index out of range
|
|
D>
|
ca_on_milton
|
|
2025-03-16 04:12:38
|
AssertionError: No councillors found
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/ca_on_milton/people.py", line 19, in scrape
assert len(councillors), "No councillors found"
AssertionError: No councillors found
|
|
C
|
ca_on_mississauga
|
2025-03-16 04:40:09
|
2025-03-16 04:40:09
|
|
|
C
|
ca_on_newmarket
|
2025-03-16 04:00:24
|
2025-03-16 04:00:25
|
|
|
C
|
ca_on_niagara
|
2025-03-16 04:14:04
|
2025-03-16 04:14:05
|
|
|
C
|
ca_on_niagara_on_the_lake
|
2025-03-16 04:08:50
|
2025-03-16 04:08:50
|
|
|
C
|
ca_on_north_dumfries
|
2025-03-16 04:47:00
|
2025-03-16 04:47:00
|
|
|
C
|
ca_on_oakville
|
2025-03-16 04:17:32
|
2025-03-16 04:17:32
|
|
|
C
|
ca_on_oshawa
|
2025-03-16 04:01:46
|
2025-03-16 04:01:46
|
|
|
D>
|
ca_on_ottawa
|
|
2025-03-16 04:42:46
|
pupa.exceptions.UnresolvedIdError: cannot resolve pseudo id to Post: ~{"label": "Orl\u00e9ans West-Innes", "organization__cl…
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 307, in do_handle
report['import'] = self.do_import(juris, args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 211, in do_import
report.update(membership_importer.import_directory(datadir))
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 190, in import_directory
return self.import_data(json_stream())
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 227, in import_data
obj_id, what = self.import_item(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 247, in import_item
data = self.prepare_for_db(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/memberships.py", line 50, in prepare_for_db
data['post_id'] = self.post_importer.resolve_json_id(data['post_id'])
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 165, in resolve_json_id
raise UnresolvedIdError(errmsg)
pupa.exceptions.UnresolvedIdError: cannot resolve pseudo id to Post: ~{"label": "Orl\u00e9ans West-Innes", "organization__classification": "legislature", "role": "Councillor"}
|
D>
04:16:19 WARNING scrapelib: sleeping for 10 seconds before retry
04:16:29 WARNING scrapelib: sleeping for 20 seconds before retry
04:16:49 WARNING scrapelib: sleeping for 40 seconds before retry
|
ca_on_peel
|
|
2025-03-16 04:17:29
|
scrapelib.HTTPError: 500 while retrieving https://services6.arcgis.com/ONZht79c8QWuX759/arcgis/rest/services/Peel_Ward_Bound…
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/utils.py", line 405, in scrape
reader = self.csv_reader(
File "/app/scrapers/utils.py", line 251, in csv_reader
response = self.get(url, **kwargs)
File "/app/scrapers/utils.py", line 198, in get
return super().get(*args, verify=kwargs.pop("verify", SSL_VERIFY), **kwargs)
File "/app/.heroku/python/lib/python3.9/site-packages/requests/sessions.py", line 602, in get
return self.request("GET", url, **kwargs)
File "/app/.heroku/python/lib/python3.9/site-packages/scrapelib/__init__.py", line 602, in request
raise HTTPError(resp)
scrapelib.HTTPError: 500 while retrieving https://services6.arcgis.com/ONZht79c8QWuX759/arcgis/rest/services/Peel_Ward_Boundary/FeatureServer/replicafilescache/Peel_Ward_Boundary_-3456469171846657907.csv
|
|
D>
|
ca_on_pickering
|
|
2025-03-16 04:09:51
|
AssertionError: No councillors found
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/ca_on_pickering/people.py", line 17, in scrape
assert len(councillors), "No councillors found"
AssertionError: No councillors found
|
|
C
|
ca_on_richmond_hill
|
2025-03-16 04:21:21
|
2025-03-16 04:21:21
|
|
|
C
|
ca_on_sault_ste_marie
|
2025-03-16 04:20:03
|
2025-03-16 04:20:03
|
|
|
C
|
ca_on_st_catharines
|
2025-03-16 04:09:16
|
2025-03-16 04:09:17
|
|
|
C
|
ca_on_thunder_bay
|
2025-03-16 04:09:49
|
2025-03-16 04:09:49
|
|
|
C
|
ca_on_toronto
|
2025-03-16 04:22:22
|
2025-03-16 04:22:22
|
|
|
C
|
ca_on_uxbridge
|
2025-03-16 04:22:33
|
2025-03-16 04:22:33
|
|
|
D>
|
ca_on_vaughan
|
|
2025-03-16 04:23:44
|
obj2 sources: []
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 307, in do_handle
report['import'] = self.do_import(juris, args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 211, in do_import
report.update(membership_importer.import_directory(datadir))
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 190, in import_directory
return self.import_data(json_stream())
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 227, in import_data
obj_id, what = self.import_item(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 265, in import_item
raise DuplicateItemError(data, obj, related.get('sources', []))
pupa.exceptions.DuplicateItemError: attempt to import data that would conflict with data already in the import: {'label': '', 'role': 'Mayor', 'person_id': 'ocd-person/92d0d43b-b0f4-455c-a4f3-a2c9e939da49', 'person_name': 'Steven Del Duca', 'organization_id': 'ocd-organization/ad2d6829-94fa-4f48-a38d-f19d251d0d54', 'post_id': 'ocd-post/b94f0812-216c-46f7-baaa-0693885a3ea9', 'on_behalf_of_id': None, 'start_date': '', 'end_date': '', 'extras': {}} (already imported as Steven Del Duca in Vaughan City Council (Mayor))
obj1 sources: []
obj2 sources: []
|
|
C
|
ca_on_waterloo
|
2025-03-16 04:22:57
|
2025-03-16 04:22:57
|
|
|
C
|
ca_on_waterloo_region
|
2025-03-16 04:46:50
|
2025-03-16 04:46:50
|
|
|
C
|
ca_on_welland
|
2025-03-16 04:20:35
|
2025-03-16 04:20:35
|
|
|
C
|
ca_on_wellesley
|
2025-03-16 04:39:39
|
2025-03-16 04:39:39
|
|
|
C
|
ca_on_whitby
|
2025-03-16 04:13:55
|
2025-03-16 04:13:55
|
|
|
C
|
ca_on_whitchurch_stouffville
|
2025-03-16 04:14:11
|
2025-03-16 04:14:11
|
|
|
C
|
ca_on_wilmot
|
2025-03-16 04:10:24
|
2025-03-16 04:10:24
|
|
|
D>
|
ca_on_windsor
|
|
2025-03-16 04:45:09
|
IndexError: list index out of range
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/ca_on_windsor/people.py", line 13, in scrape
data = json.loads(self.get(data_url).text.split(" = ")[1])
IndexError: list index out of range
|
|
D>
|
ca_on_woolwich
|
|
2025-03-16 04:46:24
|
scrapelib.HTTPError: 404 while retrieving https://www.woolwich.ca/en/council/council.asp
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/ca_on_woolwich/people.py", line 13, in scrape
page = self.lxmlize(COUNCIL_PAGE)
File "/app/scrapers/utils.py", line 217, in lxmlize
response = self.get(url, cookies=cookies, verify=verify)
File "/app/scrapers/utils.py", line 198, in get
return super().get(*args, verify=kwargs.pop("verify", SSL_VERIFY), **kwargs)
File "/app/.heroku/python/lib/python3.9/site-packages/requests/sessions.py", line 602, in get
return self.request("GET", url, **kwargs)
File "/app/.heroku/python/lib/python3.9/site-packages/scrapelib/__init__.py", line 602, in request
raise HTTPError(resp)
scrapelib.HTTPError: 404 while retrieving https://www.woolwich.ca/en/council/council.asp
|
|
D>
|
ca_pe
|
2025-03-16 04:46:14
|
2025-03-16 04:46:14
|
DETAIL: Key (division_id)=(ocd-division/country:ca/province:pe/ed:cardigan) is not present in table "opencivicdata_division…
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.9/site-packages/django/db/backends/base/base.py", line 313, in _commit
return self.connection.commit()
psycopg2.errors.ForeignKeyViolation: insert or update on table "opencivicdata_post" violates foreign key constraint "opencivicdata_post_division_id_82fef8df_fk_opencivic"
DETAIL: Key (division_id)=(ocd-division/country:ca/province:pe/ed:cardigan) is not present in table "opencivicdata_division".
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/reports/utils.py", line 75, in scrape_people
report.success_at = datetime.datetime.now(datetime.timezone.utc)
File "/app/.heroku/python/lib/python3.9/site-packages/django/db/transaction.py", line 263, in __exit__
connection.commit()
File "/app/.heroku/python/lib/python3.9/site-packages/django/utils/asyncio.py", line 26, in inner
return func(*args, **kwargs)
File "/app/.heroku/python/lib/python3.9/site-packages/django/db/backends/base/base.py", line 337, in commit
self._commit()
File "/app/.heroku/python/lib/python3.9/site-packages/django/db/backends/base/base.py", line 313, in _commit
return self.connection.commit()
File "/app/.heroku/python/lib/python3.9/site-packages/django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/app/.heroku/python/lib/python3.9/site-packages/django/db/backends/base/base.py", line 313, in _commit
return self.connection.commit()
django.db.utils.IntegrityError: insert or update on table "opencivicdata_post" violates foreign key constraint "opencivicdata_post_division_id_82fef8df_fk_opencivic"
DETAIL: Key (division_id)=(ocd-division/country:ca/province:pe/ed:cardigan) is not present in table "opencivicdata_division".
|
|
C
|
ca_pe_charlottetown
|
2025-03-16 04:17:36
|
2025-03-16 04:17:36
|
|
|
C
|
ca_pe_stratford
|
2025-03-16 04:44:02
|
2025-03-16 04:44:02
|
|
|
C
|
ca_pe_summerside
|
2025-03-16 04:14:32
|
2025-03-16 04:14:33
|
|
|
D>
|
ca_qc
|
|
2025-03-16 04:08:10
|
obj2 sources: []
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 307, in do_handle
report['import'] = self.do_import(juris, args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 209, in do_import
report.update(post_importer.import_directory(datadir))
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 190, in import_directory
return self.import_data(json_stream())
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 227, in import_data
obj_id, what = self.import_item(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 265, in import_item
raise DuplicateItemError(data, obj, related.get('sources', []))
pupa.exceptions.DuplicateItemError: attempt to import data that would conflict with data already in the import: {'label': 'Brome—Missisquoi', 'role': 'MNA', 'organization_id': 'ocd-organization/821383af-db95-44c1-9e81-12de842b56eb', 'division_id': 'ocd-division/country:ca/province:qc/ed:brome-missisquoi-1984', 'start_date': '', 'end_date': '', 'extras': {}} (already imported as MNA - Brome—Missisquoi - Assemblée nationale du Québec)
obj1 sources: []
obj2 sources: []
|
|
C
|
ca_qc_beaconsfield
|
2025-03-16 04:41:38
|
2025-03-16 04:41:38
|
|
|
D>
|
ca_qc_brossard
|
|
2025-03-16 04:43:58
|
pupa.exceptions.SameNameError: multiple people with same name "Daniel Lucier" in Jurisdiction - must provide birth_date to d…
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 307, in do_handle
report['import'] = self.do_import(juris, args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 207, in do_import
report.update(person_importer.import_directory(datadir))
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 190, in import_directory
return self.import_data(json_stream())
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 226, in import_data
for json_id, data in self._prepare_imports(data_items):
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/people.py", line 33, in _prepare_imports
raise SameNameError(name)
pupa.exceptions.SameNameError: multiple people with same name "Daniel Lucier" in Jurisdiction - must provide birth_date to disambiguate
|
|
C
|
ca_qc_cote_saint_luc
|
2025-03-16 04:43:56
|
2025-03-16 04:43:56
|
|
|
C
|
ca_qc_dollard_des_ormeaux
|
2025-03-16 04:25:18
|
2025-03-16 04:25:18
|
|
|
C
|
ca_qc_dorval
|
2025-03-16 04:19:39
|
2025-03-16 04:19:39
|
|
|
D>
|
ca_qc_gatineau
|
|
2025-03-16 04:16:15
|
pupa.exceptions.UnresolvedIdError: cannot resolve pseudo id to Post: ~{"label": "District 19", "organization__classification…
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 307, in do_handle
report['import'] = self.do_import(juris, args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 211, in do_import
report.update(membership_importer.import_directory(datadir))
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 190, in import_directory
return self.import_data(json_stream())
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 227, in import_data
obj_id, what = self.import_item(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 247, in import_item
data = self.prepare_for_db(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/memberships.py", line 50, in prepare_for_db
data['post_id'] = self.post_importer.resolve_json_id(data['post_id'])
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 165, in resolve_json_id
raise UnresolvedIdError(errmsg)
pupa.exceptions.UnresolvedIdError: cannot resolve pseudo id to Post: ~{"label": "District 19", "organization__classification": "legislature", "role": "Conseiller"}
|
D>
04:00:33 WARNING scrapelib: sleeping for 10 seconds before retry
04:00:43 WARNING scrapelib: sleeping for 20 seconds before retry
04:01:03 WARNING scrapelib: sleeping for 40 seconds before retry
04:01:43 WARNING pupa: could not save RunPlan, no successful runs of ocd-jurisdiction/country:ca/csd:2466102/legislature yet
|
ca_qc_kirkland
|
|
2025-03-16 04:01:43
|
scrapelib.HTTPError: 403 while retrieving https://www.ville.kirkland.qc.ca/portrait-municipal/conseil-municipal/elus-municip…
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/ca_qc_kirkland/people.py", line 11, in scrape
page = self.lxmlize(COUNCIL_PAGE)
File "/app/scrapers/utils.py", line 217, in lxmlize
response = self.get(url, cookies=cookies, verify=verify)
File "/app/scrapers/utils.py", line 198, in get
return super().get(*args, verify=kwargs.pop("verify", SSL_VERIFY), **kwargs)
File "/app/.heroku/python/lib/python3.9/site-packages/requests/sessions.py", line 602, in get
return self.request("GET", url, **kwargs)
File "/app/.heroku/python/lib/python3.9/site-packages/scrapelib/__init__.py", line 602, in request
raise HTTPError(resp)
scrapelib.HTTPError: 403 while retrieving https://www.ville.kirkland.qc.ca/portrait-municipal/conseil-municipal/elus-municipaux
|
|
C
|
ca_qc_laval
|
2025-03-16 04:39:36
|
2025-03-16 04:39:36
|
|
|
C
|
ca_qc_levis
|
2025-03-16 04:39:33
|
2025-03-16 04:39:33
|
|
|
C
|
ca_qc_longueuil
|
2025-03-16 04:42:43
|
2025-03-16 04:42:43
|
|
D>
04:12:41 WARNING scrapelib: sleeping for 10 seconds before retry
04:12:51 WARNING scrapelib: sleeping for 20 seconds before retry
04:13:12 WARNING scrapelib: sleeping for 40 seconds before retry
|
ca_qc_mercier
|
|
2025-03-16 04:13:52
|
scrapelib.HTTPError: 403 while retrieving https://www.ville.mercier.qc.ca/affaires-municipales/conseil-municipal/membres-du-…
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/ca_qc_mercier/people.py", line 9, in scrape
page = self.lxmlize(COUNCIL_PAGE)
File "/app/scrapers/utils.py", line 217, in lxmlize
response = self.get(url, cookies=cookies, verify=verify)
File "/app/scrapers/utils.py", line 198, in get
return super().get(*args, verify=kwargs.pop("verify", SSL_VERIFY), **kwargs)
File "/app/.heroku/python/lib/python3.9/site-packages/requests/sessions.py", line 602, in get
return self.request("GET", url, **kwargs)
File "/app/.heroku/python/lib/python3.9/site-packages/scrapelib/__init__.py", line 602, in request
raise HTTPError(resp)
scrapelib.HTTPError: 403 while retrieving https://www.ville.mercier.qc.ca/affaires-municipales/conseil-municipal/membres-du-conseil/
|
|
C
|
ca_qc_montreal
|
2025-03-16 04:11:52
|
2025-03-16 04:11:53
|
|
|
C
|
ca_qc_montreal_est
|
2025-03-16 10:49:10
|
2025-03-16 10:49:10
|
|
|
C
|
ca_qc_pointe_claire
|
2025-03-16 04:21:24
|
2025-03-16 04:21:24
|
|
|
D>
|
ca_qc_quebec
|
|
2025-03-16 04:00:30
|
pupa.exceptions.UnresolvedIdError: cannot resolve pseudo id to Post: ~{"label": "Sainte-Foy\u2013Sillery\u2013Cap-Rouge", "o…
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 307, in do_handle
report['import'] = self.do_import(juris, args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 211, in do_import
report.update(membership_importer.import_directory(datadir))
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 190, in import_directory
return self.import_data(json_stream())
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 227, in import_data
obj_id, what = self.import_item(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 247, in import_item
data = self.prepare_for_db(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/memberships.py", line 50, in prepare_for_db
data['post_id'] = self.post_importer.resolve_json_id(data['post_id'])
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 165, in resolve_json_id
raise UnresolvedIdError(errmsg)
pupa.exceptions.UnresolvedIdError: cannot resolve pseudo id to Post: ~{"label": "Sainte-Foy\u2013Sillery\u2013Cap-Rouge", "organization__classification": "legislature", "role": "Conseiller"}
|
|
C
|
ca_qc_saguenay
|
2025-03-16 04:08:39
|
2025-03-16 04:08:39
|
|
|
C
|
ca_qc_sainte_anne_de_bellevue
|
2025-03-16 04:10:21
|
2025-03-16 04:10:21
|
|
|
C
|
ca_qc_saint_jean_sur_richelieu
|
2025-03-16 04:10:52
|
2025-03-16 04:10:52
|
|
|
D>
|
ca_qc_saint_jerome
|
|
2025-03-16 04:25:21
|
AssertionError: No councillors found
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/ca_qc_saint_jerome/people.py", line 11, in scrape
assert len(councillors), "No councillors found"
AssertionError: No councillors found
|
D>
04:15:52 WARNING pupa: validation of CanadianPerson 5935b3f2-021d-11f0-8290-deb288e15ee5 failed: 1 validation errors:
Value 'data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==' for field '<obj>.image' does not match regular expression '\A(?:(?:ftp|https?)://|\Z)'
|
ca_qc_senneville
|
|
2025-03-16 04:15:52
|
Value 'data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==' for field '<obj>.image' does not match re…
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 175, in validate
validator.validate(self.as_dict(), schema)
File "/app/.heroku/python/lib/python3.9/site-packages/validictory/validator.py", line 616, in validate
raise MultipleValidationError(self._errors)
validictory.validator.MultipleValidationError: 1 validation errors:
Value 'data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==' for field '<obj>.image' does not match regular expression '\A(?:(?:ftp|https?)://|\Z)'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 104, in do_scrape
self.save_object(obj)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 89, in save_object
raise ve
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 85, in save_object
obj.validate()
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 177, in validate
raise ScrapeValueError('validation of {} {} failed: {}'.format(
pupa.exceptions.ScrapeValueError: validation of CanadianPerson 5935b3f2-021d-11f0-8290-deb288e15ee5 failed: 1 validation errors:
Value 'data:image/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==' for field '<obj>.image' does not match regular expression '\A(?:(?:ftp|https?)://|\Z)'
|
|
D>
|
ca_qc_sherbrooke
|
|
2025-03-16 04:12:15
|
pupa.exceptions.UnresolvedIdError: cannot resolve pseudo id to Post: ~{"label": "Lennoxville", "organization__classification…
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 307, in do_handle
report['import'] = self.do_import(juris, args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 211, in do_import
report.update(membership_importer.import_directory(datadir))
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 190, in import_directory
return self.import_data(json_stream())
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 227, in import_data
obj_id, what = self.import_item(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 247, in import_item
data = self.prepare_for_db(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/memberships.py", line 50, in prepare_for_db
data['post_id'] = self.post_importer.resolve_json_id(data['post_id'])
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 165, in resolve_json_id
raise UnresolvedIdError(errmsg)
pupa.exceptions.UnresolvedIdError: cannot resolve pseudo id to Post: ~{"label": "Lennoxville", "organization__classification": "legislature", "role": "Conseiller"}
|
D>
04:14:37 WARNING scrapelib: sleeping for 10 seconds before retry
04:14:47 WARNING scrapelib: sleeping for 20 seconds before retry
04:15:07 WARNING scrapelib: sleeping for 40 seconds before retry
|
ca_qc_terrebonne
|
|
2025-03-16 04:15:47
|
scrapelib.HTTPError: 403 while retrieving https://terrebonne.ca/membres-du-conseil-municipal/
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/ca_qc_terrebonne/people.py", line 9, in scrape
page = self.lxmlize(COUNCIL_PAGE, "utf-8")
File "/app/scrapers/utils.py", line 217, in lxmlize
response = self.get(url, cookies=cookies, verify=verify)
File "/app/scrapers/utils.py", line 198, in get
return super().get(*args, verify=kwargs.pop("verify", SSL_VERIFY), **kwargs)
File "/app/.heroku/python/lib/python3.9/site-packages/requests/sessions.py", line 602, in get
return self.request("GET", url, **kwargs)
File "/app/.heroku/python/lib/python3.9/site-packages/scrapelib/__init__.py", line 602, in request
raise HTTPError(resp)
scrapelib.HTTPError: 403 while retrieving https://terrebonne.ca/membres-du-conseil-municipal/
|
|
C
|
ca_qc_trois_rivieres
|
2025-03-16 04:10:18
|
2025-03-16 04:10:18
|
|
|
C
|
ca_qc_westmount
|
2025-03-16 04:09:22
|
2025-03-16 04:09:22
|
|
|
D>
|
ca_sk
|
|
2025-03-16 04:41:35
|
obj2 sources: []
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 307, in do_handle
report['import'] = self.do_import(juris, args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 209, in do_import
report.update(post_importer.import_directory(datadir))
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 190, in import_directory
return self.import_data(json_stream())
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 227, in import_data
obj_id, what = self.import_item(data)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/importers/base.py", line 265, in import_item
raise DuplicateItemError(data, obj, related.get('sources', []))
pupa.exceptions.DuplicateItemError: attempt to import data that would conflict with data already in the import: {'label': 'Saskatoon West', 'role': 'MLA', 'organization_id': 'ocd-organization/61db8858-e4ff-4767-a59c-7976d834b2ef', 'division_id': 'ocd-division/country:ca/province:sk/ed:saskatoon_west', 'start_date': '', 'end_date': '', 'extras': {}} (already imported as MLA - Saskatoon West - Legislative Assembly of Saskatchewan)
obj1 sources: []
obj2 sources: []
|
|
C
|
ca_sk_regina
|
2025-03-16 04:20:50
|
2025-03-16 04:20:50
|
|
D>
04:18:25 WARNING scrapelib: got HTTPSConnectionPool(host='saskatoonopendataconfig.blob.core.windows.net', port=443): Max retries exceeded with url: /converteddata/MayorAndCityCouncilContactInformation.csv (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5be534d700>: Failed to establish a new connection: [Errno -2] Name or service not known')) sleeping for 10 seconds before retry
04:18:35 WARNING scrapelib: got HTTPSConnectionPool(host='saskatoonopendataconfig.blob.core.windows.net', port=443): Max retries exceeded with url: /converteddata/MayorAndCityCouncilContactInformation.csv (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5be541a310>: Failed to establish a new connection: [Errno -2] Name or service not known')) sleeping for 20 seconds before retry
04:18:55 WARNING scrapelib: got HTTPSConnectionPool(host='saskatoonopendataconfig.blob.core.windows.net', port=443): Max retries exceeded with url: /converteddata/MayorAndCityCouncilContactInformation.csv (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5be541ab20>: Failed to establish a new connection: [Errno -2] Name or service not known')) sleeping for 40 seconds before retry
|
ca_sk_saskatoon
|
|
2025-03-16 04:19:35
|
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='saskatoonopendataconfig.blob.core.windows.net', port=443): Ma…
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.9/site-packages/urllib3/connection.py", line 174, in _new_conn
conn = connection.create_connection(
File "/app/.heroku/python/lib/python3.9/site-packages/urllib3/util/connection.py", line 72, in create_connection
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
File "/app/.heroku/python/lib/python3.9/socket.py", line 954, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.9/site-packages/urllib3/connectionpool.py", line 716, in urlopen
httplib_response = self._make_request(
File "/app/.heroku/python/lib/python3.9/site-packages/urllib3/connectionpool.py", line 404, in _make_request
self._validate_conn(conn)
File "/app/.heroku/python/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1061, in _validate_conn
conn.connect()
File "/app/.heroku/python/lib/python3.9/site-packages/urllib3/connection.py", line 363, in connect
self.sock = conn = self._new_conn()
File "/app/.heroku/python/lib/python3.9/site-packages/urllib3/connection.py", line 186, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f5be541ad60>: Failed to establish a new connection: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.9/site-packages/requests/adapters.py", line 667, in send
resp = conn.urlopen(
File "/app/.heroku/python/lib/python3.9/site-packages/urllib3/connectionpool.py", line 802, in urlopen
retries = retries.increment(
File "/app/.heroku/python/lib/python3.9/site-packages/urllib3/util/retry.py", line 594, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='saskatoonopendataconfig.blob.core.windows.net', port=443): Max retries exceeded with url: /converteddata/MayorAndCityCouncilContactInformation.csv (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5be541ad60>: Failed to establish a new connection: [Errno -2] Name or service not known'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/utils.py", line 405, in scrape
reader = self.csv_reader(
File "/app/scrapers/utils.py", line 251, in csv_reader
response = self.get(url, **kwargs)
File "/app/scrapers/utils.py", line 198, in get
return super().get(*args, verify=kwargs.pop("verify", SSL_VERIFY), **kwargs)
File "/app/.heroku/python/lib/python3.9/site-packages/requests/sessions.py", line 602, in get
return self.request("GET", url, **kwargs)
File "/app/.heroku/python/lib/python3.9/site-packages/scrapelib/__init__.py", line 579, in request
resp = super().request(
File "/app/.heroku/python/lib/python3.9/site-packages/scrapelib/__init__.py", line 404, in request
resp = super().request(
File "/app/.heroku/python/lib/python3.9/site-packages/scrapelib/__init__.py", line 232, in request
return super().request(
File "/app/.heroku/python/lib/python3.9/site-packages/scrapelib/__init__.py", line 175, in request
raise exception_raised
File "/app/.heroku/python/lib/python3.9/site-packages/scrapelib/__init__.py", line 122, in request
resp = super().request(
File "/app/.heroku/python/lib/python3.9/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
File "/app/.heroku/python/lib/python3.9/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
File "/app/.heroku/python/lib/python3.9/site-packages/requests/adapters.py", line 700, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='saskatoonopendataconfig.blob.core.windows.net', port=443): Max retries exceeded with url: /converteddata/MayorAndCityCouncilContactInformation.csv (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f5be541ad60>: Failed to establish a new connection: [Errno -2] Name or service not known'))
|
|
D>
|
ca_yt
|
|
2025-03-16 04:46:28
|
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://yukonassembly.ca/mlas
Traceback (most recent call last):
File "/app/reports/utils.py", line 73, in scrape_people
report.report = subcommand.handle(args, other)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 260, in handle
return self.do_handle(args, other, juris)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 305, in do_handle
report['scrape'] = self.do_scrape(juris, args, scrapers)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/cli/commands/update.py", line 173, in do_scrape
report[scraper_name] = scraper.do_scrape(**scrape_args)
File "/app/.heroku/python/lib/python3.9/site-packages/pupa/scrape/base.py", line 99, in do_scrape
for obj in self.scrape(**kwargs) or []:
File "/app/scrapers/ca_yt/people.py", line 13, in scrape
page = self.cloudscrape(COUNCIL_PAGE)
File "/app/scrapers/utils.py", line 205, in cloudscrape
response.raise_for_status()
File "/app/.heroku/python/lib/python3.9/site-packages/requests/models.py", line 1024, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://yukonassembly.ca/mlas
|