Creating LUSA data collection

[Oliph]

• Integrate the sample dataset into a collection
• Contact LUSA back to have access to their data

[BlancaCalvo]

• Look for a week with a lot of claims and a variety of topics to use for the POC

[BlancaCalvo]

• Ask for that of the month surrounding that week and for data based on the extracted keywords

[Oliph]

I did it per month

I run the following command:

db.maldita.aggregate([
{
$group: { _id: {$dateToString:  {date: {$dateFromString: {dateString:  "$createdAt"}}, format: "%Y-%m"}}, 
nbrClaims: {$sum: 1} }
}, 
{$sort: {_id: 1}}
])

{ "_id" : "2020-06", "nbrClaims" : 1 }
{ "_id" : "2021-08", "nbrClaims" : 1 }
{ "_id" : "2021-10", "nbrClaims" : 51 }
{ "_id" : "2021-11", "nbrClaims" : 52 }
{ "_id" : "2021-12", "nbrClaims" : 73 }
{ "_id" : "2022-01", "nbrClaims" : 240 }
{ "_id" : "2022-02", "nbrClaims" : 113 }
{ "_id" : "2022-03", "nbrClaims" : 193 }
{ "_id" : "2022-04", "nbrClaims" : 108 }
{ "_id" : "2022-05", "nbrClaims" : 53 }
{ "_id" : "2022-06", "nbrClaims" : 25 }

So the month with the highest numbers of claims is January 2022 but I would suggest asking for March (to avoid being between 2021 and 2022 if needed to go back slightly more in the past)

[BlancaCalvo]

Why are there no dates previous 2020 and why are there months missing?

[Oliph]

No idea, it is either their API that only started to work properly from 2022, or our script to access the data.
After checking the db, all records have a key 'createdAt' so we can discard that possibility