Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Support read opensearch_dashboards_sample_data_logs in Spark #1031

Open
penghuo opened this issue Feb 3, 2025 · 0 comments
Open
Labels
enhancement New feature or request opensearch table

Comments

@penghuo
Copy link
Collaborator

penghuo commented Feb 3, 2025

Is your feature request related to a problem?
Support query opensearch_dashboards_sample_data_logs in Spark with SQL/PPL.

  • index mappings
{
  "opensearch_dashboards_sample_data_logs": {
    "mappings": {
      "properties": {
        "@timestamp": {
          "type": "alias",
          "path": "timestamp"
        },
        "agent": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "bytes": {
          "type": "long"
        },
        "clientip": {
          "type": "ip"
        },
        "event": {
          "properties": {
            "dataset": {
              "type": "keyword"
            }
          }
        },
        "extension": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "geo": {
          "properties": {
            "coordinates": {
              "type": "geo_point"
            },
            "dest": {
              "type": "keyword"
            },
            "src": {
              "type": "keyword"
            },
            "srcdest": {
              "type": "keyword"
            }
          }
        },
        "host": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "index": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "ip": {
          "type": "ip"
        },
        "machine": {
          "properties": {
            "os": {
              "type": "text",
              "fields": {
                "keyword": {
                  "type": "keyword",
                  "ignore_above": 256
                }
              }
            },
            "ram": {
              "type": "long"
            }
          }
        },
        "memory": {
          "type": "double"
        },
        "message": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "phpmemory": {
          "type": "long"
        },
        "referer": {
          "type": "keyword"
        },
        "request": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "response": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "tags": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "timestamp": {
          "type": "date"
        },
        "url": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "utc_time": {
          "type": "date"
        }
      }
    }
  }
}

What solution would you like?
select * from opensearch.default.opensearch_dashboards_sample_data_logs should execute successfully

What alternatives have you considered?
A clear and concise description of any alternative solutions or features you've considered.

Do you have any additional context?
Add any other context or screenshots about the feature request here.

@penghuo penghuo added enhancement New feature or request untriaged and removed untriaged labels Feb 3, 2025
@penghuo penghuo changed the title [FEATURE] Support Read opensearch_dashboards_sample_data_logs in Spark [FEATURE] Support read opensearch_dashboards_sample_data_logs in Spark Feb 4, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request opensearch table
Projects
None yet
Development

No branches or pull requests

1 participant