I've URL like following
https://www.aaaa.com:5000 -> https://www.aaaa.com
https://bbbb.com:443 -> https://bbbb.com
https://cccc.com -> https://cccc.com
I need to remove only the port ..
I've tried with the following which doesn't works, it takes all the data
https://regex101.com/r/CIiALR/1
(https?://.*):(\d*)\/?(.*)`
The trick is that I must use only regex not js lib, as I need it for using Vector.
https://vector.dev/docs/reference/vrl/
Also: https://vector.dev/docs/reference/vrl/#parse-custom-logs
No need for regex, you may use the url object for this kind of work.
var url = new URL('https://www.aaaa.com:5000');
url.port = '';
console.log(url.toString());More about the url object - https://developer.mozilla.org/en-US/docs/Web/API/URL
Looking at https://vector.dev/docs/reference/vrl/ you can use a named capture group and optionally match the port number:
^(?P<withoutport>https?://[^/\s]+?)(?::\d+|$)
^ Start of string(?P<withoutport> Named group
https?:// Match the protocol[^/\s]+? Match by any char except / or a whitespace char in a non greedy way) Close named group(?::\d+|$) Match : and 1+ digits, or assert the end of the stringOr you can make it as specific as you require:
^(?P<withoutport>https?://[^/\s]+?)(?:[:?#/]\S*)?$
Depending on the version of vector you are using - I would suggest using the built in parse_url function to extract out which parts you want.
Example TOML:
[sources.my_demo_logs_source]
type = "demo_logs"
format = "shuffle"
lines = [ "https://www.aaaa.com:5000" , "https://bbbb.com:443", "https://cccc.com" ]
[transforms.my_transform]
type = "remap"
inputs = [ "my_demo_logs_source" ]
source = """
url_parts = parse_url!(.message)
.url = join!([ url_parts.scheme, url_parts.host ], "://")
"""
[sinks.my_sink_id]
type = "console"
inputs = [ "my_transform" ]
target = "stdout"
[sinks.my_sink_id.encoding]
codec = "json"
Example output:
{"message":"https://bbbb.com:443","source_type":"demo_logs","timestamp":"2022-06-28T07:20:45.202665432Z","url":"https://bbbb.com"}
{"message":"https://www.aaaa.com:5000","source_type":"demo_logs","timestamp":"2022-06-28T07:20:46.202754200Z","url":"https://www.aaaa.com"}
{"message":"https://cccc.com","source_type":"demo_logs","timestamp":"2022-06-28T07:20:47.201350600Z","url":"https://cccc.com"}
This was tested with:
❯ vector --version
vector 0.22.0 (x86_64-unknown-linux-gnu 5e937e3 2022-06-01)