Create schema validation for local Snowplow emitter
What does this MR do and why?
This MR adds local event validator to event_forward
controller.
The aim for the validator is to make sure event in development environment are valid.
This solves problem for editor extensions and other services to verify event before sending them to Snowplow collector.
This does not verifies Gitlabs events.
References
Screenshots or screen recordings
Before | After |
---|---|
How to set up and validate locally
// correct payload, should pass
curl 'http://localhost:3000/-/collect_events' \
--data-raw '{
"schema": "iglu:com.snowplowanalytics.snowplow/payload_data/jsonschema/1-0-4",
"data": [
{
"e": "pv",
"url": "http://gdk.test:3000/",
"page": "GitLab",
"refr": "http://gdk.test:3000/",
"eid": "4f1939da-210b-4463-92c3-2cdbb6a9c942",
"tv": "js-3.24.2",
"tna": "gl",
"p": "web",
"cookie": "1",
"cs": "UTF-8",
"lang": "en-GB",
"res": "1728x1117",
"cd": "30",
"tz": "Europe/Copenhagen",
"dtm": "1740062431360",
"cx": "eyJzY2hlbWEiOiJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy9jb250ZXh0cy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6W3sic2NoZW1hIjoiaWdsdTpjb20uZ2l0bGFiL2dpdGxhYl9zdGFuZGFyZC9qc29uc2NoZW1hLzEtMS0xIiwiZGF0YSI6eyJlbnZpcm9ubWVudCI6ImRldmVsb3BtZW50Iiwic291cmNlIjoiZ2l0bGFiLWphdmFzY3JpcHQiLCJjb3JyZWxhdGlvbl9pZCI6IjAxSk1ISDcxOFNFNVpYTUtUSFNGTjAzUFFTIiwicGxhbiI6bnVsbCwiZXh0cmEiOnt9LCJ1c2VyX2lkIjoxLCJnbG9iYWxfdXNlcl9pZCI6ImVSOW9DY0ZrVWxsZGZQN3hWNUk1L2tjamFXWlZTOUZsSlpYQmNNcnl1a2M9IiwiaXNfZ2l0bGFiX3RlYW1fbWVtYmVyIjpudWxsLCJuYW1lc3BhY2VfaWQiOm51bGwsInByb2plY3RfaWQiOm51bGwsImZlYXR1cmVfZW5hYmxlZF9ieV9uYW1lc3BhY2VfaWRzIjpudWxsLCJyZWFsbSI6InNlbGYtbWFuYWdlZCIsImluc3RhbmNlX2lkIjoiMmQwODU3MGQtY2ZkYi00MWYzLWI4OWUtYzcxMzliYWNlMjc1IiwiaG9zdF9uYW1lIjoiamxhcnNlbi0tMjAyMjEyMTQtUFZZNjkiLCJpbnN0YW5jZV92ZXJzaW9uIjoiMTcuOS4wIiwiY29udGV4dF9nZW5lcmF0ZWRfYXQiOiIyMDI1LTAyLTIwVDExOjE3OjIwLjczN1oiLCJnb29nbGVfYW5hbHl0aWNzX2lkIjoiIn19LHsic2NoZW1hIjoiaWdsdTpjb20uc25vd3Bsb3dhbmFseXRpY3Muc25vd3Bsb3cvd2ViX3BhZ2UvanNvbnNjaGVtYS8xLTAtMCIsImRhdGEiOnsiaWQiOiJmZDhjYTZlMy0yYmZlLTQ0MjQtODA3Zi03ODcwMGIxZjk3Y2UifX0seyJzY2hlbWEiOiJpZ2x1OmNvbS5nb29nbGUuYW5hbHl0aWNzL2Nvb2tpZXMvanNvbnNjaGVtYS8xLTAtMCIsImRhdGEiOnt9fSx7InNjaGVtYSI6ImlnbHU6b3JnLnczL1BlcmZvcm1hbmNlVGltaW5nL2pzb25zY2hlbWEvMS0wLTAiLCJkYXRhIjp7Im5hdmlnYXRpb25TdGFydCI6MTc0MDA1MDIzNTY4MiwicmVkaXJlY3RTdGFydCI6MCwicmVkaXJlY3RFbmQiOjAsImZldGNoU3RhcnQiOjE3NDAwNTAyMzU2OTcsImRvbWFpbkxvb2t1cFN0YXJ0IjoxNzQwMDUwMjM1NzEyLCJkb21haW5Mb29rdXBFbmQiOjE3NDAwNTAyMzU3MTIsImNvbm5lY3RTdGFydCI6MTc0MDA1MDIzNTcxMiwic2VjdXJlQ29ubmVjdGlvblN0YXJ0IjowLCJjb25uZWN0RW5kIjoxNzQwMDUwMjM1NzEyLCJyZXF1ZXN0U3RhcnQiOjE3NDAwNTAyMzU3MTIsInJlc3BvbnNlU3RhcnQiOjE3NDAwNTAyNDEzOTgsInJlc3BvbnNlRW5kIjoxNzQwMDUwMjQxNDA0LCJ1bmxvYWRFdmVudFN0YXJ0IjoxNzQwMDUwMjQxNDE4LCJ1bmxvYWRFdmVudEVuZCI6MTc0MDA1MDI0MTQxOCwiZG9tTG9hZGluZyI6MTc0MDA1MDI0MTQzNSwiZG9tSW50ZXJhY3RpdmUiOjE3NDAwNTAyNDE3OTgsImRvbUNvbnRlbnRMb2FkZWRFdmVudFN0YXJ0IjoxNzQwMDUwMjUxOTE3LCJkb21Db250ZW50TG9hZGVkRXZlbnRFbmQiOjE3NDAwNTAyNTE5MTgsImRvbUNvbXBsZXRlIjoxNzQwMDUwMjUyMDI4LCJsb2FkRXZlbnRTdGFydCI6MTc0MDA1MDI1MjAyOCwibG9hZEV2ZW50RW5kIjoxNzQwMDUwMjUyMDI4fX1dfQ",
"vp": "1728x992",
"ds": "1713x1038",
"vid": "2",
"sid": "4e39265b-81a1-493d-8d62-faa54e35fe48",
"duid": "654a2c38-edc9-4d34-95bf-c192f7262dd5",
"stm": "1740135995404"
}
]
}'
// context has error - incorrect data type assigned for `plan` property (I set it manually to `false`)
curl 'http://localhost:3000/-/collect_events' \
--data-raw '{
"schema": "iglu:com.snowplowanalytics.snowplow/payload_data/jsonschema/1-0-4",
"data": [
{
"e": "pv",
"url": "http://gdk.test:3000/",
"page": "GitLab",
"refr": "http://gdk.test:3000/",
"eid": "4f1939da-210b-4463-92c3-2cdbb6a9c942",
"tv": "js-3.24.2",
"tna": "gl",
"p": "web",
"cookie": "1",
"cs": "UTF-8",
"lang": "en-GB",
"res": "1728x1117",
"cd": "30",
"tz": "Europe/Copenhagen",
"dtm": "1740062431360",
"cx": "eyJzY2hlbWEiOiJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy9jb250ZXh0cy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6W3sic2NoZW1hIjoiaWdsdTpjb20uZ2l0bGFiL2dpdGxhYl9zdGFuZGFyZC9qc29uc2NoZW1hLzEtMS0xIiwiZGF0YSI6eyJlbnZpcm9ubWVudCI6ImRldmVsb3BtZW50Iiwic291cmNlIjoiZ2l0bGFiLWphdmFzY3JpcHQiLCJjb3JyZWxhdGlvbl9pZCI6IjAxSk1ISDcxOFNFNVpYTUtUSFNGTjAzUFFTIiwicGxhbiI6IGZhbHNlLCJleHRyYSI6e30sInVzZXJfaWQiOjEsImdsb2JhbF91c2VyX2lkIjoiZVI5b0NjRmtVbGxkZlA3eFY1STUva2NqYVdaVlM5RmxKWlhCY01yeXVrYz0iLCJpc19naXRsYWJfdGVhbV9tZW1iZXIiOm51bGwsIm5hbWVzcGFjZV9pZCI6bnVsbCwicHJvamVjdF9pZCI6bnVsbCwiZmVhdHVyZV9lbmFibGVkX2J5X25hbWVzcGFjZV9pZHMiOm51bGwsInJlYWxtIjoic2VsZi1tYW5hZ2VkIiwiaW5zdGFuY2VfaWQiOiIyZDA4NTcwZC1jZmRiLTQxZjMtYjg5ZS1jNzEzOWJhY2UyNzUiLCJob3N0X25hbWUiOiJqbGFyc2VuLS0yMDIyMTIxNC1QVlk2OSIsImluc3RhbmNlX3ZlcnNpb24iOiIxNy45LjAiLCJjb250ZXh0X2dlbmVyYXRlZF9hdCI6IjIwMjUtMDItMjBUMTE6MTc6MjAuNzM3WiIsImdvb2dsZV9hbmFseXRpY3NfaWQiOiIifX0seyJzY2hlbWEiOiJpZ2x1OmNvbS5zbm93cGxvd2FuYWx5dGljcy5zbm93cGxvdy93ZWJfcGFnZS9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6eyJpZCI6ImZkOGNhNmUzLTJiZmUtNDQyNC04MDdmLTc4NzAwYjFmOTdjZSJ9fSx7InNjaGVtYSI6ImlnbHU6Y29tLmdvb2dsZS5hbmFseXRpY3MvY29va2llcy9qc29uc2NoZW1hLzEtMC0wIiwiZGF0YSI6e319LHsic2NoZW1hIjoiaWdsdTpvcmcudzMvUGVyZm9ybWFuY2VUaW1pbmcvanNvbnNjaGVtYS8xLTAtMCIsImRhdGEiOnsibmF2aWdhdGlvblN0YXJ0IjoxNzQwMDUwMjM1NjgyLCJyZWRpcmVjdFN0YXJ0IjowLCJyZWRpcmVjdEVuZCI6MCwiZmV0Y2hTdGFydCI6MTc0MDA1MDIzNTY5NywiZG9tYWluTG9va3VwU3RhcnQiOjE3NDAwNTAyMzU3MTIsImRvbWFpbkxvb2t1cEVuZCI6MTc0MDA1MDIzNTcxMiwiY29ubmVjdFN0YXJ0IjoxNzQwMDUwMjM1NzEyLCJzZWN1cmVDb25uZWN0aW9uU3RhcnQiOjAsImNvbm5lY3RFbmQiOjE3NDAwNTAyMzU3MTIsInJlcXVlc3RTdGFydCI6MTc0MDA1MDIzNTcxMiwicmVzcG9uc2VTdGFydCI6MTc0MDA1MDI0MTM5OCwicmVzcG9uc2VFbmQiOjE3NDAwNTAyNDE0MDQsInVubG9hZEV2ZW50U3RhcnQiOjE3NDAwNTAyNDE0MTgsInVubG9hZEV2ZW50RW5kIjoxNzQwMDUwMjQxNDE4LCJkb21Mb2FkaW5nIjoxNzQwMDUwMjQxNDM1LCJkb21JbnRlcmFjdGl2ZSI6MTc0MDA1MDI0MTc5OCwiZG9tQ29udGVudExvYWRlZEV2ZW50U3RhcnQiOjE3NDAwNTAyNTE5MTcsImRvbUNvbnRlbnRMb2FkZWRFdmVudEVuZCI6MTc0MDA1MDI1MTkxOCwiZG9tQ29tcGxldGUiOjE3NDAwNTAyNTIwMjgsImxvYWRFdmVudFN0YXJ0IjoxNzQwMDUwMjUyMDI4LCJsb2FkRXZlbnRFbmQiOjE3NDAwNTAyNTIwMjh9fV19",
"vp": "1728x992",
"ds": "1713x1038",
"vid": "2",
"sid": "4e39265b-81a1-493d-8d62-faa54e35fe48",
"duid": "654a2c38-edc9-4d34-95bf-c192f7262dd5",
"stm": "1740135995404"
}
]
}'
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Related to #561995
Edited by Niko Belokolodov