CSV meter upload validates values

[huss]

Is your feature request related to a problem? Please describe.

The create meter page stops admins from entering bad values by constraining choices and/or checking before saving. It would be good if the CSV meter upload page did similar checks.

Describe the solution you'd like

See above.

Describe alternatives you've considered

No checking

Additional context

None

[mikedivine]

I will work on this issue.

[ethanernst11]

I can work on this issue.

[huss]

I can work on this issue.

I think this is okay since @mikedivine is not working on this at this time. If you need any help on this then let me know.

[ethanernst11]

I have a quick question. Where can I find the code for the create meter page that stops admins from entering bad values?

[huss]

I have a quick question. Where can I find the code for the create meter page that stops admins from entering bad values?

See src/client/app/components/meters/CreateMeterModalComponent.tsx (there is an edit equivalent). There are a number of checks here that are done in different ways (different ways is not needed for this issue). As examples, it validates GPS, can't have cumulative reset if not cumulative, set identifier if name given, no blank name, .... Let me know if you need anything else.

[ethanernst11]

Is the csv meter upload also creating a meter?

[huss]

Is the csv meter upload also creating a meter?

In general, the CSV meter upload creates meter(s). It is possible to edit a single meter but that is less common. If you have not seen it yet, you may find the documentation at https://openenergydashboard.org/helpV1_0_0/adminCsvImport/ useful. It is for v1.0.0 so is a little out of date with development but still useful.

[ethanernst11]

I apologize but this ended up being more complicated than expected and I do not have enough time currently to work on it. So I will not be able to continue working on this issue for finish it. I am sorry for any inconvenience.

[huss]

I apologize but this ended up being more complicated than expected and I do not have enough time currently to work on it. So I will not be able to continue working on this issue for finish it. I am sorry for any inconvenience.

It's fine and thanks for letting us know. If you want to work on something in the future then please just let us know (and we can help find a task that suits your situation if that helps).

[SageMar]

If it's alright, @cmatthews444 and I will tackle this issue

[huss]

@SageMar & @cmatthews444 I assigned SageMar this issue since I cannot do the other GitHub ID. Thanks for wanting to work on this. There are a number of desired tests and getting any of them done would be nice. Please let me know if I can help.

[cmatthews444]

@huss @SageMar and I have done some digging and found these that appear still need to be implemented.

1. no negative meter readings (min value is 0)
2. check for max value
3. reading variation
4. file size

Do we need the multiple min/max values being imported into CreateMeterModalComponent.tsx from: import {
MAX_DATE, MAX_DATE_MOMENT,
MAX_ERRORS, MAX_VAL, MIN_DATE,
MIN_DATE_MOMENT, MIN_VAL,
isValidCreateMeter,
selectCreateMeterUnitCompatibility,
selectDefaultCreateMeterValues
} from '../../redux/selectors/adminSelectors';

Another small possible thing to fix is the error message for incorrect gps name says "For meter name" and could point to the meter name for easier fixing.

On top of these were there any other validations that you saw that needed to be implanted?

[huss]

@cmatthews444 & @SageMar Thanks for looking into this issue. I think either the issue was not clear or we are thinking about different things. When I first go to the meter create page for admins I see:

The fields in red are required. If I try to upload a meter CSV file without a name or type then I get a failure message from the database. It would probably be better to validate the values earlier and report it before submission. In addition, on the meter create page there are drop down menus for type so only valid ones can be entered. In an ideal world the values in the CSV would also be checked to make sure it is one of the allowed ones. There are a number of values that are limited but the CSV comes in as text so it would have to be checked.

Also, looking at src/client/app/components/meters/CreateMeterModalComponent.tsx, it does several other checks:

• Validate the GPS value
• Checks that displayable is okay via the form checks. There are other invalid checks on the form.
• Dynamically updates certain menus (such as the allowed unit) to make sure it is valid
• area is >= 0 and there is an area unit if there is an area value

I have not listed or found them all but this is what OED was hoping to accomplish with this issue. I know there are a lot and details so if you want to discuss more or start with a subset that would be fine. It may be that have a call together would help.

As for the checks you specified, I'm not sure if they are needed and would want to discuss more to be certain we are thinking about the same thing. For example, you can have a negative reading in OED (as of v1.0.0).

As for your question about needing the import, I suspect you will need some of those values to set if not in the CSV or check that the values are okay. Now, if the database automatically sets the value correctly then it might be okay to allow an empty value on the CSV file because the database fixes it up.

As for the error message, it does seems that the name of the meter would be valuable after the second colon. I have not looked to see what needs to be changed but I assume the meter name is available to add. If you want to look at this and need any help then let me know.

[SageMar]

@huss Thanks for that clarification! I do think think we're both still a little confused about a couple things, so if you have time to do a meeting to chat real quick that would be super helpful. Are you free at all on Wednesday? We're both free that whole day, so whatever time would work for you or we can try scheduling a different day.

[huss]

@SageMar Would either 11:00 or 1:00 PST on Wednesday, 11/27 work for you? Could we meet on the OED Discord server in the General voice channel? If neither times work then we can look for others that day. If you prefer something other than Discord then that is also okay.

[SageMar]

@huss 11:00 PST on 11/27 works great! We'll both join the discord voice at that time. Thanks for taking the time, looking forward to chatting with you!