Support NGINX Security Monitor experience for NAP WAF customers

[brianehlert]

NGINX Ingress Controller (NIC) supports NGINX App Protect Web Application Firewall (NAP WAF).
To augment the NAP WAF experience the Security Monitor (SM) module was constructed to provide visibility into violations as well as WAF policy construction, signature and campaign management.

This full cycle experience is enabled through a component called NGINX Agent (agent) that runs in the same process space as NGINX and the NAP WAF module.
NIC currently does not include nor support agent, due primarily to historic assumptions of the agent implementation for different use cases.
The agent has added some capabilities to change its behavior related to NIC concerns but it is not fully understood if this meets all requirements of NIC and needs to be investigated.

This should achieve three primary high goals:

• embedding Agent into NIC images (including starting/stopping, configuring)
• NIC instances are visible in the NIM UI (general metrics)
• NIC + NAP WAF customers can view violations and build Policy Bundles that can be applied to NIC via CI/CD pipeline

AC:

• investigate embedding agent into the NIC + NAP WAF image(s)
• understand the options necessary to run agent in a way that ensures safety for NIC
  • Report only (do not allow configuration push that might create a two sources of truth state)
  • how to configure NAP WAF to route logs to agent
• understand how to configure agent to connect to SM after NIC is deployed and restart agent
  • agent should not be started if it is not configured
  • when configuration is provided via YAML it should be applied and agent started/restarted
• Implement decisions to achieve the complete NAP WAF experience for customers
• NIC instances should "register" with NIM and be visible in the UI

Not Criteria:

• this will support using agent to pull and apply any configuration directly from SM
• any security policy change should be applied through exporting a security Policy bundle and apply that using the Policy bundle capability of NIC

### Tasks
- [ ] https://github.com/nginxinc/kubernetes-ingress/issues/4887
- [ ] https://github.com/nginxinc/kubernetes-ingress/issues/4978
- [ ] https://github.com/nginxinc/kubernetes-ingress/issues/4981
- [ ] https://github.com/nginxinc/kubernetes-ingress/issues/4987
- [ ] https://github.com/nginxinc/kubernetes-ingress/issues/5145
- [ ] https://github.com/nginxinc/kubernetes-ingress/issues/5241
- [ ] https://github.com/nginxinc/kubernetes-ingress/issues/5242
- [ ] https://github.com/nginxinc/kubernetes-ingress/issues/5339

Aha! Link: https://nginx.aha.io/epics/NIC-E-122