[Melbourne-pm] Extracting a parenthesized fragment from a string

[Robert Norris]

Hi all,

I'm currently fixing a few bugs in Mail::IMAPClient that for whatever
reason have gone unnoticed. One of these is that its fetch_hash() method
can't handle a parenthesized value if contains parentheses. In other
words, its not dealing with nested parentheses correctly.

A typical server response to a FETCH command might look something like this:

  * 2 FETCH (UID 2 FLAGS (\Answered \Seen) INTERNALDATE "05-Jun-2008 07:19:01 +1000" RFC822.SIZE 5144 BODYSTRUCTURE ("text" "plain" ("charset" "ISO-8859-1") NIL NIL "7bit" 2053 46 NIL ("inline" NIL) NIL NIL))

fetch_hash() seeks to turn that into the following hash:

  "UID"           => "2",
  "FLAGS"         => "\Answered \Seen",
  "INTERNALDATE"  => "05-Jun-2008 07:19:01 +1000",
  "RFC822.SIZE"   => "5144",
  "BODYSTRUCTURE" => "text" "plain" ("charset" "ISO-8859-1") NIL NIL "7bit" 2053 46 NIL ("inline" NIL) NIL NIL",

In other words, the response data consists of key/value pairs. The
values are either simple strings (matched by \S+), quoted strings
(matched by "[^"]*") or a parenthesized fragment, which may include
parentheses itself.

What's inside the parentheses is actually completely opaque as far as
this method is concerned. The original code used \([\)]*\) to try and
match it, which works great until there is a closing paren inside.

My attempt is based on mjd's regex to match balanced parentheses here:

  http://perl.plover.com/yak/regex/samples/slide083.html

This is what I have so far for matching/capturing the parenthesized
fragment.  Its actually part of a larger regex that matches the other
stuff described above, but this doesn't work in isolation either so I
don't think those things are affecting it:

  (?:
    (?{ local $d=0 }) # set depth to 0
    (                 # start capture
      (?:                  
        \(            # opening paren
        (?{$d++})     # increment the depth
        |  
        \)            # or closing paren
        (?{$d--})     # decrement the depth
        (?(?{$d==0})  # if we're back to the start
          (*ACCEPT)   # we're done
        )
        |
        (?>[^()]*)    # or there's just some normal text
      )*
      (?(?{$d!=0})    # done. did we reach a matching closing paren
        (?!)          # nope, failed
      )
    )                 # end capture
  )
 
Running a basic test program with -Mre=debug seems to suggest that this
is doing something close to what I want, stopping when it gets to the
correct closing paren. The only problem is that nothing gets captured,
even though the description of (*ACCEPT) in perlre makes it sound like
it should.

So, what's wrong here? And is there an easier way to do this? Note that
I'm restricted to Perl 5.6, which sucks as I'd really like to try the
PARNO stuff that came with 5.10.

Thanks,
Rob.