Blueprint not working

[Frazou1]

Bug Description

Since update the blueprint to 1.3.7 nothing trigger the automation, but if i look into frigate, i got event

Version: 1.3.7

Service Call

alias: Notif AI Cam Sonette
description: ""
use_blueprint:
  path: valentinfrlch/event_summary.yaml
  input:
    mode: Frigate
    notify_device:
      - d0d9e52e51bf00ca693a6161e29b1b6a
    camera_entities:
      - camera.cam_sonette
    required_zones:
      - cam sonette motion
    object_type:
      - person
      - car
    tap_navigate: "{{video if video != '''' else image}}"
    provider: 01JHNP36WMNRXSF2WYRTKT704X
    temperature: 0.3

Logs

Paste relevant logs here

Additional context

[Lemonhead-Snickers]

Same issue. My old notifications based on the SgtBatten blueprint work fine. Doesn't make much sense given that they both fire on frigate/events

[airbornetrooper82573]

I'm just now joining this adventure and same for m e. I tried the beta still not working. I tried the SGTBatten blueprint and I got a notification right away. First I thought maybe something was wrong with the HA app on my iPhone.

[valentinfrlch]

Could someone with this issue post traces?

[Lemonhead-Snickers]

Could someone with this issue post traces?

I would love to, but I just installed two days ago and I've only run the new blueprint. As a result, my automation has never triggered. See below that the SgtBatten blueprint has triggered, and I'll post the trace for that.

trace automation.frigate_notifications_0_12_0_4c 2025-01-27T22_09_09.569034+00_00.json

Edit: on the off chance the issue was having both automations enabled at the same time and using the same trigger (mqtt frigate/events), I turned off the SgtBatten version and just ran the 1.3.7 blueprint. No luck there.

[airbornetrooper82573]

Could someone with this issue post traces?

I've never gotten it to actually run. When I do try to make it run I get this error in logs:

Logger: homeassistant.components.automation.ai_event_summary_llm_vision_v1_3_7
Source: components/automation/init.py:664
integration: Automation (documentation, issues)
First occurred: 8:23:06 AM (1 occurrences)
Last logged: 8:23:06 AM

Error rendering variables: UndefinedError: 'dict object' has no attribute 'payload_json'

[valentinfrlch]

The issue is likely the updated frigate trigger:

trigger: mqtt
topic: frigate/events
id: frigate_trigger
enabled: "{{mode == 'Frigate'}}"

The enabled template should evaluate to true only if mode is set to "Frigate". Previously there was an issue that would throw an error if mqtt wasn't installed even though camera mode doesn't need mqtt.

[Frazou1]

Same thing here, i cant have trace because they never run. If you want to try something i can !

[popler]

+1

[Lemonhead-Snickers]

The issue is likely the updated frigate trigger:

trigger: mqtt
topic: frigate/events
id: frigate_trigger
enabled: "{{mode == 'Frigate'}}"
The enabled template should evaluate to true only if mode is set to "Frigate". Previously there was an issue that would throw an error if mqtt wasn't installed even though camera mode doesn't need mqtt.

I may troubleshoot based on this a bit.

Did the MQTT payload trigger change in this version 1.3.7? It looks like SgtBatten listens for the event type "new," while LLM Vision listens for the event type "end." Wondering if that could be the issue. Will try that too.

[Lemonhead-Snickers]

Sorry to keep updating, but could it be that "mode:" as a blueprint input variable is conflicting with HomeAssistant's "mode:" setting?

[Frazou1]

Sorry to keep updating, but could it be that "mode:" as a blueprint input variable is conflicting with HomeAssistant's "mode:" setting?

Im not sure how i can do, but can we test a basic automation without the blueprint to see if is working ?

[airbornetrooper82573]

Im not sure how i can do, but can we test a basic automation without the blueprint to see if is working ?

You can do that under developer tools > actions
https://llm-vision.gitbook.io/examples/examples/scripts

[Lemonhead-Snickers]

The issue is likely the updated frigate trigger:

trigger: mqtt
topic: frigate/events
id: frigate_trigger
enabled: "{{mode == 'Frigate'}}"
The enabled template should evaluate to true only if mode is set to "Frigate". Previously there was an issue that would throw an error if mqtt wasn't installed even though camera mode doesn't need mqtt.

Confirmed, in part. By removing any condition based on mode == 'Frigate', the automation fires and listens on frigate/events. Note that line 306 in the blueprint sets mode: single, which probably conflicts. However, I've broken it somehow in my config because the trace is failing at the value template when I go walk in front of the camera. Not sure what's going on there, but will investigate more as I have time.

Here's what I changed it to:

trigger:
  - platform: mqtt
    topic: frigate/events
    id: frigate_trigger
  - platform: state
    entity_id: !input camera_entities
    to: !input trigger_state
    id: camera_trigger
  - platform: state
    entity_id: !input motion_sensors
    to: "on"
    id: motion_sensor_trigger
condition:
  - condition: template
    value_template: >
      {{ trigger.payload_json["type"] == "new"
        and ('camera.' + trigger.payload_json['after']['camera']|lower) in camera_entities_list
        and ((object_types_list|length) == 0 or ((trigger.payload_json['after']['label']|lower) in object_types_list))
        and (not required_zones_list or ((set(trigger.payload_json['after']['current_zones']).intersection(required_zones_list))))
        and not (ignore_stationary and trigger.payload_json['after']['stationary'])
      }}

Note that it also wasn't triggering when type was set to end.

[popler]

I also have error in log Error rendering variables: UndefinedError: 'dict object' has no attribute 'payload_json'

Even that automation was not triggered.

[Frazou1]

The issue is likely the updated frigate trigger:
trigger: mqtt
topic: frigate/events
id: frigate_trigger
enabled: "{{mode == 'Frigate'}}"
The enabled template should evaluate to true only if mode is set to "Frigate". Previously there was an issue that would throw an error if mqtt wasn't installed even though camera mode doesn't need mqtt.

Confirmed, in part. By removing any condition based on mode == 'Frigate', the automation fires and listens on frigate/events. Note that line 306 in the blueprint sets mode: single, which probably conflicts. However, I've broken it somehow in my config because the trace is failing at the value template when I go walk in front of the camera. Not sure what's going on there, but will investigate more as I have time.

Here's what I changed it to:

trigger:

• platform: mqtt
  topic: frigate/events
  id: frigate_trigger
• platform: state
  entity_id: !input camera_entities
  to: !input trigger_state
  id: camera_trigger
• platform: state
  entity_id: !input motion_sensors
  to: "on"
  id: motion_sensor_trigger
  condition:
• condition: template
  value_template: >
  {{ trigger.payload_json["type"] == "new"
  and ('camera.' + trigger.payload_json['after']['camera']|lower) in camera_entities_list
  and ((object_types_list|length) == 0 or ((trigger.payload_json['after']['label']|lower) in object_types_list))
  and (not required_zones_list or ((set(trigger.payload_json['after']['current_zones']).intersection(required_zones_list))))
  and not (ignore_stationary and trigger.payload_json['after']['stationary'])
  }}
  Note that it also wasn't triggering when type was set to end.

Have you change it into the blueprint directly ?

[Frazou1]

Im not sure how i can do, but can we test a basic automation without the blueprint to see if is working ?

You can do that under developer tools > actions https://llm-vision.gitbook.io/examples/examples/scripts

They have only example for camera mode not frigate :(

[efranulic]

I installed 2 days ago and first I made an automation triggred by a door open, and it works fine. Then I installed the blueprint because I wanted to use it with frigate, but it won't trigger...

[aserper]

+1 - having the same issue. Automation doesn't trigger.
I also can't add a trace because it never fires. When I'm hitting the save button after configuring the blueprint I am getting this error

[tvofi]

+1 have the same issue. Automation has not triggered since updating blueprint to 1.3.7. Beta blueprint also affected. Tried downgrading blueprint to 1.3.5 which restored trigger functionality, but instead errored when sending OpenAI api request. Could be that 1.3.5 is not compatible with the latest version of the integration?

[Frazou1]

@valentinfrlch Does you need some info to investigate ?

[jarjarfinks]

I also did what @tvofi did to no avail, except I'm using Ollama. Any idea what's happening?

[valentinfrlch]

The issue is likely the updated frigate trigger:

trigger: mqtt
topic: frigate/events
id: frigate_trigger
enabled: "{{mode == 'Frigate'}}"
The enabled template should evaluate to true only if mode is set to "Frigate". Previously there was an issue that would throw an error if mqtt wasn't installed even though camera mode doesn't need mqtt.

Yes. This is what causes the problem.
While I work on a fix, you can switch to camera mode and add the occupancy binary_sensors to the blueprint. They are automatically created if you have the Frigate integration installed.

[valentinfrlch]

I think I have found a way to solve this issue. The hotfix is live in the beta blueprint. Feedback is welcome!

To install the beta blueprint, add this as source:

https://github.com/valentinfrlch/ha-llmvision/blob/main/blueprints/event_summary_beta.yaml

[aserper]

I tried it, while I'm not getting that error message I mentioned above, the automation still doesn't trigger. I'm attaching a trace

trace automation.ai_event_summary_beta_test 2025-01-30T22_03_22.558344+00_00.json

[popler]

I just tested this new version with Frigate trigger. It's almost working for me. Automation is triggered. I receive notification on selected phone with description from AI Open. But... I have black video in notification. Blueprint had to send correct data to OpenAI , because I have right description, but wrong file is attached to notification.

[UhtredTheBold]

The beta blueprint is working for me

[airbornetrooper82573]

Tried the beta, still seem to get the same error:

Logger: homeassistant.components.automation.ai_event_summary_beta_test
Source: components/automation/init.py:664
integration: Automation (documentation, issues)
First occurred: 8:19:50 AM (1 occurrences)
Last logged: 8:19:50 AM

Error rendering variables: UndefinedError: 'dict object' has no attribute 'payload_json'

[efranulic]

I installed the new beta blueprint, but the automation still don't trigger

[UhtredTheBold]

Make sure to reload the Automation part of your yaml setup. You can do so in the developer settings > Yaml. Alternatively you can also restart HA.

Yes, I did it, but still nothing...

Just to make sure as it's not been explicitly mentioned, you do have to create a new automation using the beta blueprint.

[efranulic]

Make sure to reload the Automation part of your yaml setup. You can do so in the developer settings > Yaml. Alternatively you can also restart HA.

Yes, I did it, but still nothing...

Just to make sure as it's not been explicitly mentioned, you do have to create a new automation using the beta blueprint.

Yeah, that's what I did. I deleted the old one and create a new automation based on beta blueprint

[airbornetrooper82573]

The beta blueprint is working for me

Can you share all what you entered in to make the blueprint work for you? I've tried using required zones and not as well as object types. I'm only trying frigate and not camera

[UhtredTheBold]

The beta blueprint is working for me

Can you share all what you entered in to make the blueprint work for you? I've tried using required zones and not as well as object types. I'm only trying frigate and not camera

I'm using frigate with a single camera, a single required zone and object type set to person.

I have a single notify device, the remember option enabled and i'm sending to openAI. All other settings are left at default.

[airbornetrooper82573]

alias: AI Event Summary (Beta) TEST description: "" use_blueprint: path: valentinfrlch/event_summary_beta TEST.yaml input: input_mode: Frigate notify_device: - e33d9524cb5e239a24919bb6df25f23b camera_entities: - camera.defender_driveway_2 required_zones: - Driveway object_type: - person tap_navigate: /dashboard-cameras provider: 01JJN69CQCD2980TE54YHZQBAT model: llava-phi3:latest
Still throws the same error as before when I run action and check the logs.

[jarjarfinks]

I'm also having the error, even in the beta version. I tried removing the multiple camera entities used for frigate to only use one camera but same result.

…

Message ID: ***@***.***>

[Frazou1]

I import the beta and create new automation. Automation are now trigger and look working fine with zone and object. Sometine i receive false positive and i got no snapshot or live preview. But the only thing remain is Ollama look like they doesnt answer. I only receive Dog seen or Person seen. I look into the log and i got this : 2025-01-31 13:03:05.029 ERROR (MainThread) [homeassistant.components.automation.notification_cam_ext_arriere_ai] Notification Cam Ext Arriere AI: Analyze event: choice 1: Error executing script. Error for call_service at pos 1: Request failed:

[airbornetrooper82573]

I've tried again. Still same error for me. I have Frigate integration installed from HACS, version 5.7.0. Not sure if that matters or not. I use docker version of HA so I don't have add-ons.

[popler]

Scrit is tiggering with beta version. I found errors about fetching mp4 file. Sometimes it is working ok.

AI Event Summary (Beta) frigate: Analyze event: choice 1: Error executing script. Error for call_service at pos 1: Failed to fetch frigate clip 1738400903.938686-twtxpc
AI Event Summary (Beta) frigate: Error executing script. Error for choose at pos 4: Failed to fetch frigate clip 1738400903.938686-twtxpc

I just tested this new version with Frigate trigger. It's almost working for me. Automation is triggered. I receive notification on selected phone with description from AI Open. But... I have black video in notification. Blueprint had to send correct data to OpenAI , because I have right description, but wrong file is attached to notification.

[TiddlyWiddly]

1 camera through frigate and never fires, using the beta.

[lukaszzyla]

Hi!
It is the same for me.
I had some luck with the 1.30 but since then I remember going through 1.33 1.35 1.36 and 1.37 it is not working.
In one of the older versions it was being triggered but the AI description was cut off after a word or two.
Not being triggered on 1.37 and betas...

[valentinfrlch]

1 camera through frigate and never fires, using the beta.

Well it did fire, otherwise you wouldn't see a trace. Did you add the camera to the blueprint?

[TiddlyWiddly]

1 camera through frigate and never fires, using the beta.

Well it did fire, otherwise you wouldn't see a trace. Did you add the camera to the blueprint?

I added the frigate camera yes. It (the notification) does not fire. The only thing that fires is the trigger from the events MQTT. Some other template setting is consistently stopping it.

[valentinfrlch]

Can you share a trace? The main issue (automation doesn't trigger) seems to be resolved however.

[airbornetrooper82573]

I see the regular blueprint was updated recently. Both do not work and I'm still getting the same error.

[valentinfrlch]

Please share traces if you want to report a bug.

Also, unless you have exactly the same problem as someone else who has already created an issue, create a new issue. Posting different bugs in the same issue leads to confusion.

[jlai79]

Automation is triggered:

but:

[lukaszzyla]

for me it is triggered also now on LLM 1.38 and blueprint 1.3.8 but fails:

[vw-kombi]

I only came across this blueprint today, and hence have the latest of everything installed. I Spent last 2 hours chasing my tail but I cant get it to fire at all. Then I came across this and it sounds the same issue. The other Frigate Notifications blueprint is running fine, so I know I have Frigate and Mqtt etc etc all set up correctly. I have tried so much and finally needed to o a trace, which is new to me. I suspect I have the same as the lukaszzyla above. Here is some info -

My mode is set to Frigate
My device is set - iPhone14-VW
My camera entities are set to two cams - side-cam and front-cam (I tried one or the other)
Required zones are currently blank, but I tried all options there. not sure if these are REQUIRED or optional.
Included object types - I tried all options there. not sure if these are REQUIRED or optional.

I figure all the rest is irrelevant as its not firing.

In case the image does not work above, the first step is this -

trigger: mqtt
topic: frigate/events
id: frigate_trigger
enabled: '{{listen_mqtt}}'

Then the next a/b is this -

Executed: 8 February 2025 at 15:10:03
Result:
result: false
entities: []

with this in the step config -

condition: template
value_template: |
{% if input_mode == 'Frigate' %}
{{ trigger.payload_json["type"] == "end"
and ('camera.' + trigger.payload_json['after']['camera']|lower) in camera_entities_list
and ((object_types_list|length) == 0 or ((trigger.payload_json['after']['label']|lower) in object_types_list))
and (not required_zones_list or ((set(trigger.payload_json['after']['current_zones']).intersection(required_zones_list))))
and not (ignore_stationary and trigger.payload_json['after']['stationary'])
}}
{% else %}
true
{% endif %}

Current Blueprint config if this helps (I have tried shedloads of different options -

id: '1738990945520'
alias: Blueprint - AI Event Summary (LLM Vision v1.3.8)
description: ''
use_blueprint:
path: valentinfrlch/event_summary.yaml
input:
input_mode: Frigate
notify_device:
- f82feb5cd2d0b3b680ef3f841ae85004
camera_entities:
- camera.side_cam
- camera.front_cam
provider: 01JKHQ0300SXD7YBXQA380GNAF

Further to this, Id like to add - I tried the beta and it also did the exact same thing. I restarted HA after deleting the original and blueprint, I also set one cam, one zone, person like another post above - nothing works..

[tvofi]

for me it is triggered also now on LLM 1.38 and blueprint 1.3.8 but fails:

Same for me, it triggers but always fails on that condition.

[vw-kombi]

That’s a few of us then.
I’m happy to test anything needed if I can help in any way.

[valentinfrlch]

For now I suggest switching to the camera mode. If you have the frigate integration installed you should see some occupancy sensors (binary_sensor) which you can use as triggers in the blueprint.

[vw-kombi]

I messed with this all morning, and the camera / occupancy sensor thing does trigger now, and gets more down the trace, and while I thought it was not working for the sending, it in fact was working (just was not popping up on my iphone).

On a side not, the latest version of frigate popped up (0.15) 30 mins ago, so I installed that, and it has all the AI stuff built into it so I am moving focus to that for now - it is working well and adding its own description to events. Notifications from that could make your blueprint obsolete I expect.

I will come back to yours once the frigate stuff it again working.

[valentinfrlch]

The blueprint has become complicated, and many people have had problems with it. I haven't upgraded Frigate to 0.15, but I think having the summaries in Frigate directly is easier to set up and use.

I don't think it'll make the blueprint obsolete, but perhaps the "Frigate" mode is unnecessary now.
Removing the Frigate mode and focussing on Camera mode for the people who don't have Frigate would make maintaining the blueprint much easier and less prone to failure.

[Frazou1]

For now I suggest switching to the camera mode. If you have the frigate integration installed you should see some occupancy sensors (binary_sensor) which you can use as triggers in the blueprint.

Im looking to switch camera mode but quick question does i need to select the 4 binary create by home assistant for my specific camera ?

Also in the trigger state is set for recording. But its look like when a event finish status stay at recording.

[valentinfrlch]

You need to select one binary_sensor for each camera (if you have multiple). The blueprint will be triggered when the binary_sensor switches on. If you select person it'll only trigger when frigate detects a person etc. The order of the sensors must be the same as the cameras.

[Frazou1]

I would like to thank you for your help. I was able to do automation witj camera mode and binary sensor frigate. I have 2 last question. Is it possible to add a wait time before sending notification in the blueprint ? Why ? Because sometime they need a little delay to provide the summerrize or to the video be available. Also, i have a couple of camera and some camera looking at same place but different angle. Does the blueprint can detect that and summarize only one of them ?

[valentinfrlch]

A beta for v1.4.0 of both the blueprint and the integration is out now. The blueprint has been restructured significantly to fix these issues.
You you can try out the beta version of the blueprint by importing event_summary_beta.yaml. You'll also need to update to the v1.4.0-beta.1 as the blueprint is not backwards compatible.

If you have feedback or suggestions, please join the discussion here: #213