Python JSON append if value doesn't exist

Question

I've got a json file with 30-ish, blocks of "dicts" where every block has and ID, like this:

{
      "ID": "23926695",
      "webpage_url": "https://.com",
      "logo_url": null,
      "headline": "aewafs",
      "application_deadline": "2020-03-31T23:59:59",
}

Since my script pulls information in the same way from an API more than once, I would like to append new "blocks" to the json file only if the ID doesn't already exist in the JSON file.

I've got something like this so far:

import os

check_empty = os.stat('pbdb.json').st_size
if check_empty == 0:
    with open('pbdb.json', 'w') as f:
        f.write('[\n]')    # Writes '[' then linebreaks with '\n' and writes ']'
output = json.load(open("pbdb.json"))

for i in jobs:
    output.append({
        'ID': job_id, 
        'Title': jobtitle, 
        'Employer' : company, 
        'Employment type' : emptype, 
        'Fulltime' : tid, 
        'Deadline' : deadline, 
        'Link' : webpage
    })

with open('pbdb.json', 'w') as job_data_file:
    json.dump(output, job_data_file)

but I would like to only do the "output.append" part if the ID doesn't exist in the Json file.

Answer 1

I am not able to complete the code you provided but I added an example to show how you can achieve the none duplicate list of jobs(hopefully it helps):

# suppose `data` is you input data with duplicate ids included
data = [{'id': 1, 'name': 'john'}, {'id': 1, 'name': 'mary'}, {'id': 2, 'name': 'george'}]

# using dictionary comprehension you can eliminate the duplicates and finally get the results by calling the `values` method on dict.
noduplicate = list({itm['id']:itm for itm in data}.values())

with open('pbdb.json', 'w') as job_data_file:
    json.dump(noduplicate, job_data_file)

Answer 2

I'll just go with a database guys, thank you for your time, we can close this thread now