Continued from https://forums.oracle.com/ords/apexds/post/introduction-to-regular-expressions-continued-9561, here's the third and final part of my introduction to regular expressions. As always, if you find mistakes or have examples that you think could be solved through regular expressions, please post them.
Having fun with regular expressions - Part 3
In some cases, I may have to search for different values in the same column. If the searched values are fixed, I can use the logical OR operator or the IN clause, like in this example (using my brute force data generator from part 2):
SELECT data
FROM TABLE(regex_utils.gen_data('abcxyz012', 4))
WHERE data IN ('abc', 'xyz', '012');
There are of course some workarounds as presented in this asktom thread but for a quick solution, there's of course an alternative approach available. Remember the "|" pipe symbol as OR operator inside regular expressions? Take a look at this:
SELECT data
FROM TABLE(regex_utils.gen_data('abcxyz012', 4))
WHERE REGEXP_LIKE(data, '^(abc|xyz|012)$')
;
I can even use strings composed of values like
'abc, xyz , 012'
by simply using another regular expression to replace "," and spaces with the "|" pipe symbol. After reading part 1 and 2 that shouldn't be too hard, right? Here's my "thinking in regular expression": Replace every "," and 0 or more leading/trailing spaces.
Ready to try your own solution?
Does it look like this?
SELECT data
FROM TABLE(regex_utils.gen_data('abcxyz012', 4))
WHERE REGEXP_LIKE(data, '^(' || REGEXP_REPLACE('abc, xyz , 012', ' *, *', '|') || ')$')
;
If I wouldn't use the "^" and "$" metacharacter, this SELECT would search for any occurence inside the data column, which could be useful if I wanted to combine LIKE and IN clause. Take a look at this example where I'm looking for 'abc%', 'xyz%' or '012%' and adding a case insensitive match parameter to it:
SELECT data
FROM TABLE(regex_utils.gen_data('abcxyz012', 4))
WHERE REGEXP_LIKE(data, '^(abc|xyz|012)', 'i')
;
An equivalent non regular expression solution would have to look like this, not mentioning other options with adding an extra "," and using the INSTR function:
SELECT data
FROM (SELECT data, LOWER(DATA) search
FROM TABLE(regex_utils.gen_data('abcxyz012', 4))
)
WHERE search LIKE 'abc%'
OR search LIKE 'xyz%'
OR search LIKE '012%'
;
SELECT data
FROM (SELECT data, SUBSTR(LOWER(DATA), 1, 3) search
FROM TABLE(regex_utils.gen_data('abcxyz012', 4))
)
WHERE search IN ('abc', 'xyz', '012')
;
I'll leave it to your imagination how a complete non regular example with
'abc, xyz , 012'
as search condition would look like.
As mentioned in the first part, regular expressions are not very good at formatting, except for some selected examples, such as phone numbers, which in my demonstration, have different formats. Using regular expressions, I can change them to a uniform representation:
WITH t AS (SELECT '123-4567' phone
FROM dual
UNION
SELECT '01 345678'
FROM dual
UNION
SELECT '7 87 8787'
FROM dual
)
SELECT t.phone, REGEXP_REPLACE(REGEXP_REPLACE(phone, '[^0-9]'), '(.{3})(.*)', '(\1)-\2')
FROM t
;
First, all non digit characters are beeing filtered, afterwards the remaining string is put into a "(xxx)-xxxx" format, but not cutting off any phone numbers that have more than 7 digits. Using such a conversion could also be used to check the validity of entered data, and updating the value with a uniform format afterwards.
Thinking about it, why not use regular expressions to check other values about their formats? How about an IP4 address? I'll do this step by step, using 127.0.0.1 as the final test case.
First I want to make sure, that each of the 4 parts of an IP address remains in the range between 0-255. Regular expressions are good at string matching but they don't allow any numeric comparisons. What valid strings do I have to take into consideration?
Single digit values: 0-9
Double digit values: 00-99
Triple digit values: 000-199, 200-255 (this one will be the trickiest part)
So far, I will have to use the "|" pipe operator to match all of the allowed combinations. I'll use my brute force generator to check if my solution works for a single value:
SELECT data
FROM TABLE(regex_utils.gen_data('0123456789', 3))
WHERE REGEXP_LIKE(data, '^(25[0-5]|2[0-4][0-9]|[01]?[0-9]{1,2})$')
;
More than 255 records? Leading zeros are allowed, but checking on all the records, there's no value above 255. First step accomplished. The second part is to make sure, that there are 4 such values, delimited by a "." dot. So I have to check for 0-255 plus a dot 3 times and then check for another 0-255 value. Doesn't sound to complicated, does it?
Using first my brute force generator, I'll check if I've missed any possible combination:
SELECT data
FROM TABLE(regex_utils.gen_data('03.', 15))
WHERE REGEXP_LIKE(data,
'^((25[0-5]|2[0-4][0-9]|[01]?[0-9]{1,2})\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9]{1,2})$'
)
;
Looks good to me. Let's check on some sample data:
WITH t AS (SELECT '127.0.0.1' ip
FROM dual
UNION
SELECT '256.128.64.32'
FROM dual
)
SELECT t.ip
FROM t WHERE REGEXP_LIKE(t.ip,
'^((25[0-5]|2[0-4][0-9]|[01]?[0-9]{1,2})\.){3}(25[0-5]|2[0-4][0-9]|[01]?[0-9]{1,2})$'
)
;
No surprises here. I can take this example a bit further and try to format valid addresses to a uniform representation, as shown in the phone number example. My goal is to display every ip address in the "xxx.xxx.xxx.xxx" format, using leading zeros for 2 and 1 digit values.
Regular expressions don't have any format models like for example the TO_CHAR function, so how could this be achieved? Thinking in regular expressions, I first have to find a way to make sure, that each single number is at least three digits wide. Using my example, this could look like this:
WITH t AS (SELECT '127.0.0.1' ip
FROM dual
)
SELECT t.ip, REGEXP_REPLACE(t.ip, '([0-9]+)(\.?)', '00\1\2')
FROM t
;
Look at this: leading zeros. However, that first value "00127" doesn't look to good, does it? If you thought about using a second regular expression function to remove any excess zeros, you're absolutely right. Just take the past examples and think in regular expressions. Did you come up with something like this?
WITH t AS (SELECT '127.0.0.1' ip
FROM dual
)
SELECT t.ip, REGEXP_REPLACE(REGEXP_REPLACE(t.ip, '([0-9]+)(\.?)', '00\1\2'),
'[0-9]*([0-9]{3})(\.?)', '\1\2'
)
FROM t
;
Think about the possibilities: Now you can sort a table with unformatted IP addresses, if that is a requirement in your application or you find other values where you can use that "trick".
Since I'm on checking INET (internet) type of values, let's do some more, for example an e-mail address. I'll keep it simple and will only check on the
"x@x.xxx", "x@x.xxxx" and "x@x.xx.xx" format, where x represents an alphanumeric character. If you want, you can look up the corresponding RFC definition and try to build your own regular expression for that one.
Now back to this one: At least one alphanumeric character followed by an "@" at sign which is followed by at least one alphanumeric character followed by a "." dot and exactly 3 more alphanumeric characters or 2 more characters followed by a "." dot and another 2 characters. This should be an easy one, right? Use some sample e-mail addresses and my brute force generator, you should be able to verify your solution.
Here's mine:
SELECT data
FROM TABLE(regex_utils.gen_data('a1@.', 9))
WHERE REGEXP_LIKE(data, '^[[:alnum:]]+@[[:alnum:]]+(\.[[:alnum:]]{3,4}|(\.[[:alnum:]]{2}){2})$', 'i');
Checking on valid domains, in my opinion, should be done in a second function, to keep the checks by itself simple, but that's probably a discussion about readability and taste.
How about checking a valid URL? I can reuse some parts of the e-mail example and only have to decide what type of URLs I want, for example "http://", "https://" and "ftp://", any subdomain and a "/" after the domain. Using the case insensitive match parameter, this shouldn't take too long, and I can use this thread's URL as a test value. But take a minute to figure that one out for yourself.
Does it look like this?
WITH t AS (SELECT '437109 URL
FROM dual
UNION
SELECT 'http://x/'
FROM dual
)
SELECT t.URL
FROM t
WHERE REGEXP_LIKE(t.URL, '^(https*|ftp)://(.+\.)*[[:alnum:]]+(\.[[:alnum:]]{3,4}|(\.[[:alnum:]]{2}){2})/', 'i')
;
Update: Improvements in 10g2
All of you, who are using 10g2 or XE (which includes some of 10g2 features) may want to take a look at several improvements in this version. First of all, there are new, perl influenced meta characters.
Rewriting my example from the first lesson, the WHERE clause would look like this:
WHERE NOT REGEXP_LIKE(t.col1, '^\d+$')
Or my example with searching decimal numbers:
'^(\.\d+|\d+(\.\d*)?)$'
Saves some space, doesn't it? However, this will only work in 10g2 and future releases.
Some of those meta characters even include non matching lists, for example "\S" is equivalent to "[^ ]", so my example in the second part could be changed to:
SELECT NVL(LENGTH(REGEXP_REPLACE('Having fun with regular expressions', '\S')), 0)
FROM dual
;
Other meta characters support search patterns in strings with newline characters. Just take a look at the link I've included.
Another interesting meta character is "?" non-greedy. In 10g2, "?" not only means 0 or 1 occurrence, it means also the first occurrence. Let me illustrate with a simple example:
SELECT REGEXP_SUBSTR('Having fun with regular expressions', '^.* +')
FROM dual
;
This is old style, "greedy" search pattern, returning everything until the last space.
SELECT REGEXP_SUBSTR('Having fun with regular expressions', '^.* +?')
FROM dual
;
In 10g2, you'd get only "Having " because of the non-greedy search operation. Simulating that behavior in 10g1, I'd have to change the pattern to this:
SELECT REGEXP_SUBSTR('Having fun with regular expressions', '^[^ ]+ +')
FROM dual
;
Another new option is the "x" match parameter. It's purpose is to ignore whitespaces in the searched string. This would prove useful in ignoring trailing/leading spaces for example. Checking on unsigned integers with leading/trailing spaces would look like this:
SELECT REGEXP_SUBSTR(' 123 ', '^[0-9]+$', 1, 1, 'x')
FROM dual
;
However, I've to be careful. "x" would also allow " 1 2 3 " to qualify as valid string.
I hope you enjoyed reading this introduction and hope you'll have some fun with using regular expressions.
C.
Fixed some typos ...
Message was edited by:
cd
Included 10g2 features
Message was edited by:
cd
[Edited by BluShadow 13/03/2023: Fix links in new forum platform]