[go: up one dir, main page]

Skip to content

Create schema validation for local Snowplow emitter

What does this MR do and why?

This MR adds local event validator to event_forward controller.

The aim for the validator is to make sure event in development environment are valid.

This solves problem for editor extensions and other services to verify event before sending them to Snowplow collector.

This does not verifies Gitlabs events.

References

Screenshots or screen recordings

Before After

How to set up and validate locally

// correct payload, should pass

curl 'http://localhost:3000/-/collect_events' \
--data-raw '{
    "schema": "iglu:com.snowplowanalytics.snowplow/payload_data/jsonschema/1-0-4",
    "data": [
      {
        "e": "pv",
        "url": "http://gdk.test:3000/",
        "page": "GitLab",
        "refr": "http://gdk.test:3000/",
        "eid": "4f1939da-210b-4463-92c3-2cdbb6a9c942",
        "tv": "js-3.24.2",
        "tna": "gl",
        "p": "web",
        "cookie": "1",
        "cs": "UTF-8",
        "lang": "en-GB",
        "res": "1728x1117",
        "cd": "30",
        "tz": "Europe/Copenhagen",
        "dtm": "1740062431360",
        "cx": "eyJzY2hlbWEiOiJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy9jb250ZXh0cy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6W3sic2NoZW1hIjoiaWdsdTpjb20uZ2l0bGFiL2dpdGxhYl9zdGFuZGFyZC9qc29uc2NoZW1hLzEtMS0xIiwiZGF0YSI6eyJlbnZpcm9ubWVudCI6ImRldmVsb3BtZW50Iiwic291cmNlIjoiZ2l0bGFiLWphdmFzY3JpcHQiLCJjb3JyZWxhdGlvbl9pZCI6IjAxSk1ISDcxOFNFNVpYTUtUSFNGTjAzUFFTIiwicGxhbiI6bnVsbCwiZXh0cmEiOnt9LCJ1c2VyX2lkIjoxLCJnbG9iYWxfdXNlcl9pZCI6ImVSOW9DY0ZrVWxsZGZQN3hWNUk1L2tjamFXWlZTOUZsSlpYQmNNcnl1a2M9IiwiaXNfZ2l0bGFiX3RlYW1fbWVtYmVyIjpudWxsLCJuYW1lc3BhY2VfaWQiOm51bGwsInByb2plY3RfaWQiOm51bGwsImZlYXR1cmVfZW5hYmxlZF9ieV9uYW1lc3BhY2VfaWRzIjpudWxsLCJyZWFsbSI6InNlbGYtbWFuYWdlZCIsImluc3RhbmNlX2lkIjoiMmQwODU3MGQtY2ZkYi00MWYzLWI4OWUtYzcxMzliYWNlMjc1IiwiaG9zdF9uYW1lIjoiamxhcnNlbi0tMjAyMjEyMTQtUFZZNjkiLCJpbnN0YW5jZV92ZXJzaW9uIjoiMTcuOS4wIiwiY29udGV4dF9nZW5lcmF0ZWRfYXQiOiIyMDI1LTAyLTIwVDExOjE3OjIwLjczN1oiLCJnb29nbGVfYW5hbHl0aWNzX2lkIjoiIn19LHsic2NoZW1hIjoiaWdsdTpjb20uc25vd3Bsb3dhbmFseXRpY3Muc25vd3Bsb3cvd2ViX3BhZ2UvanNvbnNjaGVtYS8xLTAtMCIsImRhdGEiOnsiaWQiOiJmZDhjYTZlMy0yYmZlLTQ0MjQtODA3Zi03ODcwMGIxZjk3Y2UifX0seyJzY2hlbWEiOiJpZ2x1OmNvbS5nb29nbGUuYW5hbHl0aWNzL2Nvb2tpZXMvanNvbnNjaGVtYS8xLTAtMCIsImRhdGEiOnt9fSx7InNjaGVtYSI6ImlnbHU6b3JnLnczL1BlcmZvcm1hbmNlVGltaW5nL2pzb25zY2hlbWEvMS0wLTAiLCJkYXRhIjp7Im5hdmlnYXRpb25TdGFydCI6MTc0MDA1MDIzNTY4MiwicmVkaXJlY3RTdGFydCI6MCwicmVkaXJlY3RFbmQiOjAsImZldGNoU3RhcnQiOjE3NDAwNTAyMzU2OTcsImRvbWFpbkxvb2t1cFN0YXJ0IjoxNzQwMDUwMjM1NzEyLCJkb21haW5Mb29rdXBFbmQiOjE3NDAwNTAyMzU3MTIsImNvbm5lY3RTdGFydCI6MTc0MDA1MDIzNTcxMiwic2VjdXJlQ29ubmVjdGlvblN0YXJ0IjowLCJjb25uZWN0RW5kIjoxNzQwMDUwMjM1NzEyLCJyZXF1ZXN0U3RhcnQiOjE3NDAwNTAyMzU3MTIsInJlc3BvbnNlU3RhcnQiOjE3NDAwNTAyNDEzOTgsInJlc3BvbnNlRW5kIjoxNzQwMDUwMjQxNDA0LCJ1bmxvYWRFdmVudFN0YXJ0IjoxNzQwMDUwMjQxNDE4LCJ1bmxvYWRFdmVudEVuZCI6MTc0MDA1MDI0MTQxOCwiZG9tTG9hZGluZyI6MTc0MDA1MDI0MTQzNSwiZG9tSW50ZXJhY3RpdmUiOjE3NDAwNTAyNDE3OTgsImRvbUNvbnRlbnRMb2FkZWRFdmVudFN0YXJ0IjoxNzQwMDUwMjUxOTE3LCJkb21Db250ZW50TG9hZGVkRXZlbnRFbmQiOjE3NDAwNTAyNTE5MTgsImRvbUNvbXBsZXRlIjoxNzQwMDUwMjUyMDI4LCJsb2FkRXZlbnRTdGFydCI6MTc0MDA1MDI1MjAyOCwibG9hZEV2ZW50RW5kIjoxNzQwMDUwMjUyMDI4fX1dfQ",
        "vp": "1728x992",
        "ds": "1713x1038",
        "vid": "2",
        "sid": "4e39265b-81a1-493d-8d62-faa54e35fe48",
        "duid": "654a2c38-edc9-4d34-95bf-c192f7262dd5",
        "stm": "1740135995404"
      }
    ]
  }'

// context has error - incorrect data type assigned for `plan` property (I set it manually to `false`)

curl 'http://localhost:3000/-/collect_events' \
--data-raw '{
"schema": "iglu:com.snowplowanalytics.snowplow/payload_data/jsonschema/1-0-4",
  "data": [
    {
      "e": "pv",
      "url": "http://gdk.test:3000/",
      "page": "GitLab",
      "refr": "http://gdk.test:3000/",
      "eid": "4f1939da-210b-4463-92c3-2cdbb6a9c942",
      "tv": "js-3.24.2",
      "tna": "gl",
      "p": "web",
      "cookie": "1",
      "cs": "UTF-8",
      "lang": "en-GB",
      "res": "1728x1117",
      "cd": "30",
      "tz": "Europe/Copenhagen",
      "dtm": "1740062431360",
      "cx": "eyJzY2hlbWEiOiJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy9jb250ZXh0cy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6W3sic2NoZW1hIjoiaWdsdTpjb20uZ2l0bGFiL2dpdGxhYl9zdGFuZGFyZC9qc29uc2NoZW1hLzEtMS0xIiwiZGF0YSI6eyJlbnZpcm9ubWVudCI6ImRldmVsb3BtZW50Iiwic291cmNlIjoiZ2l0bGFiLWphdmFzY3JpcHQiLCJjb3JyZWxhdGlvbl9pZCI6IjAxSk1ISDcxOFNFNVpYTUtUSFNGTjAzUFFTIiwicGxhbiI6IGZhbHNlLCJleHRyYSI6e30sInVzZXJfaWQiOjEsImdsb2JhbF91c2VyX2lkIjoiZVI5b0NjRmtVbGxkZlA3eFY1STUva2NqYVdaVlM5RmxKWlhCY01yeXVrYz0iLCJpc19naXRsYWJfdGVhbV9tZW1iZXIiOm51bGwsIm5hbWVzcGFjZV9pZCI6bnVsbCwicHJvamVjdF9pZCI6bnVsbCwiZmVhdHVyZV9lbmFibGVkX2J5X25hbWVzcGFjZV9pZHMiOm51bGwsInJlYWxtIjoic2VsZi1tYW5hZ2VkIiwiaW5zdGFuY2VfaWQiOiIyZDA4NTcwZC1jZmRiLTQxZjMtYjg5ZS1jNzEzOWJhY2UyNzUiLCJob3N0X25hbWUiOiJqbGFyc2VuLS0yMDIyMTIxNC1QVlk2OSIsImluc3RhbmNlX3ZlcnNpb24iOiIxNy45LjAiLCJjb250ZXh0X2dlbmVyYXRlZF9hdCI6IjIwMjUtMDItMjBUMTE6MTc6MjAuNzM3WiIsImdvb2dsZV9hbmFseXRpY3NfaWQiOiIifX0seyJzY2hlbWEiOiJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy93ZWJfcGFnZS9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6eyJpZCI6ImZkOGNhNmUzLTJiZmUtNDQyNC04MDdmLTc4NzAwYjFmOTdjZSJ9fSx7InNjaGVtYSI6ImlnbHU6Y29tLmdvb2dsZS5hbmFseXRpY3MvY29va2llcy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6e319LHsic2NoZW1hIjoiaWdsdTpvcmcudzMvUGVyZm9ybWFuY2VUaW1pbmcvanNvbnNjaGVtYS8xLTAtMCIsImRhdGEiOnsibmF2aWdhdGlvblN0YXJ0IjoxNzQwMDUwMjM1NjgyLCJyZWRpcmVjdFN0YXJ0IjowLCJyZWRpcmVjdEVuZCI6MCwiZmV0Y2hTdGFydCI6MTc0MDA1MDIzNTY5NywiZG9tYWluTG9va3VwU3RhcnQiOjE3NDAwNTAyMzU3MTIsImRvbWFpbkxvb2t1cEVuZCI6MTc0MDA1MDIzNTcxMiwiY29ubmVjdFN0YXJ0IjoxNzQwMDUwMjM1NzEyLCJzZWN1cmVDb25uZWN0aW9uU3RhcnQiOjAsImNvbm5lY3RFbmQiOjE3NDAwNTAyMzU3MTIsInJlcXVlc3RTdGFydCI6MTc0MDA1MDIzNTcxMiwicmVzcG9uc2VTdGFydCI6MTc0MDA1MDI0MTM5OCwicmVzcG9uc2VFbmQiOjE3NDAwNTAyNDE0MDQsInVubG9hZEV2ZW50U3RhcnQiOjE3NDAwNTAyNDE0MTgsInVubG9hZEV2ZW50RW5kIjoxNzQwMDUwMjQxNDE4LCJkb21Mb2FkaW5nIjoxNzQwMDUwMjQxNDM1LCJkb21JbnRlcmFjdGl2ZSI6MTc0MDA1MDI0MTc5OCwiZG9tQ29udGVudExvYWRlZEV2ZW50U3RhcnQiOjE3NDAwNTAyNTE5MTcsImRvbUNvbnRlbnRMb2FkZWRFdmVudEVuZCI6MTc0MDA1MDI1MTkxOCwiZG9tQ29tcGxldGUiOjE3NDAwNTAyNTIwMjgsImxvYWRFdmVudFN0YXJ0IjoxNzQwMDUwMjUyMDI4LCJsb2FkRXZlbnRFbmQiOjE3NDAwNTAyNTIwMjh9fV19",
      "vp": "1728x992",
      "ds": "1713x1038",
      "vid": "2",
      "sid": "4e39265b-81a1-493d-8d62-faa54e35fe48",
      "duid": "654a2c38-edc9-4d34-95bf-c192f7262dd5",
      "stm": "1740135995404"
    }
  ]
}'

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #561995

Edited by Niko Belokolodov

Merge request reports

Loading