[Raleigh-talk] [Perl help] - Multiple substitutions in a file

Wed Feb 23 05:07:55 PST 2011

On Wed, Feb 23, 2011 at 1:51 AM, John Ricker <sephtin+pm-talk at gmail.com>wrote:

> Very nice...
> I assumed that it'd have to be done via some sort of loop structure.. never
> crossed my mind that I could shift through the array to accomplish this.  :)
>
> Sitting in front of the code, I agree that it would work.  The marker does
> not change, the replacements are already in an array, and the file could
> easily be grabbed in a string.
> Thanks again for the assistance!
>
> This is too easy.  :(
>

s/too easy/perl/

:)

mike

> I'll see if I can find something more fun and difficult.. like maybe
> instead of having to call system ("xxd...") to convert the binary to hex,
> see if I can find a way to do it in Perl as well.. if so, then I think I've
> eliminated the last "system" call in my script that can possibly
> be eliminated... ;)
>
> Anyway, thanks again!
> -John
>
> On Wed, Feb 23, 2011 at 2:05 AM, Mike South <msouth at gmail.com> wrote:
>
>> If you have the whole file in a string and the replacements in an array
>> and the 'marker' doesn't change, I think the following will work.
>>
>> The /e modifier executes code, so s/foo/ do something here /eg will
>> replace each instance of foo with whatever 'do something here', executed as
>> perl code, returns.  So I just made it grab the next color off the list and
>> put that in place.
>>
>> In case you have more of the 'markers' than you have replacements, I just
>> put $1 back in right where we found it.  The stuff that's printing out the
>> length of the string is a rough check to make sure I'm not accidentally
>> clobbering part of the string.
>>
>> mike
>>
>>
>> use strict;
>> use warnings;
>>
>> my $file = '
>> 08 00 00 00 FF FF FF FF FF FF FF FF 1B 00 00 00 02 01 10 00 28 01 00 00 09
>> 00 00 00 FF FF FF FF FF FF FF FF 1C 00 00 00 14 00 14 00 0D 00 00 00 00 00
>> 0D 00 18 00 00 00 11 00 00 00 FF FF FF FF 08 00 00 05 02 0E 00 00 18 00 00
>> 00 0E 00 00 00 FF FF FF FF 08 00 00 1C 00 00 00 FF 18 00 00 00 09 00 00 00
>> FF FF FF FF 08 00 00 11 10 00 00 00 18 00 00 00 0F 00 00 00 FF FF FF FF 08
>> 00 00 01 11 02 02 01 18 00 00 00 03 00 00 00 08 00 00 1C FF FF FF FF 08 00
>> 00 01 05 00 08 01 18 00 00 00 10 00 00 00 FF FF FF FF 08 00 00 05 01 0F 00
>> 00 18 00 00 00 05 00 08 00 00 1C 00 00 FF FF FF FF 08 00 00 05 01 0F 00 08
>> 00 00 1C 00 18 00 00 ';
>>
>> my $length = length $file;
>>
>> my $color1 = '11#AA#BB#CC';
>> my $color2 = '22#DD#EE#FF';
>> my $color3 = '33#XX#XX#XX';
>> my @replacements = ($color1, $color2, $color3);
>> #my @replacements = ($color1, $color2);
>>
>> my $marker = '08 00 00 1C ';
>> my $xx = qr/[[:xdigit:]]{2}/; # two hex digits
>> $file =~ s/$marker(($xx ){3}$xx)/$marker . (shift(@replacements) ||
>> $1)/ge;
>>
>>
>> print $file,$/;
>> print "length was $length, now it's ", length($file), $/;
>>
>>
>> On Tue, Feb 22, 2011 at 11:49 PM, Mike South <msouth at gmail.com> wrote:
>>
>>> Also those \d's aren't going to match A-F are they?
>>>
>>> Are you familiar with qr// to let you store a regex bit in a variable?
>>>  That might make some of this more readable.
>>>
>>> mike
>>>
>>> On Tue, Feb 22, 2011 at 10:58 PM, Matt Nash <mattnashbrowns at gmail.com>wrote:
>>>
>>>> Of course, you want actual spaces (not '\s') in the replace string, but
>>>> you knew that.
>>>>
>>>>
>>>> On Tue, Feb 22, 2011 at 9:56 PM, Matt Nash <mattnashbrowns at gmail.com>wrote:
>>>>
>>>>> Hi John,
>>>>>
>>>>> You are reading the entire file into an array of lines, then processing
>>>>> that array using a C-style for loop and interpolating the index var into the
>>>>> search string... and that is the LEAST crazy thing. :)  It works, of course,
>>>>> because it turns out that there are infinitely many ways to do it.
>>>>>
>>>>> For your immediate problem of the search-and-replace, I recommend
>>>>> reading the  whole file into a single string, newlines and all, then using
>>>>> the /gc modifiers in the regex.  Maybe put your colorChange stuff into an
>>>>> array that you loop through, checking the regex for the nth match:
>>>>>
>>>>> @changes = ($change1, $change2, $change3);
>>>>>
>>>>> foreach $change (@changes) {
>>>>>     $wholefile =~
>>>>> s|08\s00\s00\s1C\s\d\d\s\d\d\s\d\d\s\d\d|08\s00\s00\s1C\s$change|gc;
>>>>> }
>>>>>
>>>>> see http://perldoc.perl.org/perlre.html and
>>>>> http://perldoc.perl.org/perlretut.htm<http://perldoc.perl.org/perlretut.html#Using-regular-expressions-in-Perl> for
>>>>> lots of details, but the upshot of the modifiers is: g makes it keep looking
>>>>> for matches; c tells it to remember where it last matched, and start from
>>>>> there on the next match.  You don't have to care what n is, because you are
>>>>> matching your search string exactly as many times as you have changes to
>>>>> make.
>>>>>
>>>>> ...but this project sounds ripe for refactoring, if I may be so bold.
>>>>>  Could it be that what you really need is a templating system?
>>>>>
>>>>> Thanks for bringing some much-needed questions to this list!
>>>>>
>>>>> Matt
>>>>>
>>>>> On Tue, Feb 22, 2011 at 8:31 PM, John Ricker <
>>>>> sephtin+pm-talk at gmail.com> wrote:
>>>>>
>>>>>> Received so much help with the last question (which seems to be
>>>>>> functioning great, btw.. thanks all)... thought I'd try again.  ;)
>>>>>>
>>>>>> Have a script that was recently migrated to Perl from shell/bash, and
>>>>>> am wondering if there's an easy way in Perl to do the following-
>>>>>>
>>>>>> File(s) being modified are hex, and look something like:
>>>>>> ---x---
>>>>>> ...
>>>>>> 08 00 00 00 FF FF FF FF FF FF FF FF 1B 00 00 00 02 01 10 00 28 01 00
>>>>>> 00 09 00 00 00 FF FF FF FF FF FF FF FF 1C 00 00 00 14 00 14 00 0D 00 00 00
>>>>>> 00 00 0D 00 18 00 00 00 11 00 00 00 FF FF FF FF 08 00 00 05 02 0E 00 00 18
>>>>>> 00 00 00 0E 00 00 00 FF FF FF FF 08 00 00 1C 00 00 00 FF 18 00 00 00 09 00
>>>>>> 00 00 FF FF FF FF 08 00 00 11 10 00 00 00 18 00 00 00 0F 00 00 00 FF FF FF
>>>>>> FF 08 00 00 01 11 02 02 01 18 00 00 00 03 00 00 00 FF FF FF FF 08 00 00 01
>>>>>> 05 00 08 01 18 00 00 00 10 00 00 00 FF FF FF FF 08 00 00 05 01 0F 00 00 18
>>>>>> 00 00 00 05 00 00 00 FF FF FF FF 08 00 00 05 01 0F 00 00 18 00 00
>>>>>> ...
>>>>>> ---x---
>>>>>> File(s) contain several occurrences of "08 00 00 1c ## ## ## ##" or
>>>>>> more appropriately for this list: "08\s00\s00\s1C\s\d\d\s\d\d\s\d\d\s\d\d"
>>>>>>
>>>>>>  Now on to the good stuff...  I would like to replace the ## ## ## ##
>>>>>> with my chosen colors, that are being provided by vars... Example:
>>>>>> colorChange1="FF 00 00 FF"
>>>>>> colorChange2="FF FF 00 FF"
>>>>>> colorChange3="FF 00 FF FF"
>>>>>> ...
>>>>>>
>>>>>> I'm wondering how it might be possible, to replace the
>>>>>> FIRST occurrence (in the file, NOT in a line) of "08 00 00 1C ## ## ## ##"
>>>>>> with "08 00 00 1C $colorChange1", the second with "08 00 00 1C
>>>>>> $colorChange2", etc.
>>>>>>
>>>>>> More info:
>>>>>> --Script modifies files to theme them, in this case, I'm taking a
>>>>>> BINARY (compiled XML) file, converting it to HEX via xxd, and then changing
>>>>>> the text color for a theme via substitution.
>>>>>> --I originally thought it could be done similar to what sed does...
>>>>>> with s|(08\s00\s00\s1C)\s\d\d\s\d\d\s\d\d\s\d\d|$1$colorChange1|1  (note
>>>>>> last digit...), but testing has shown that this doesn't work as expected.
>>>>>> --Not really related, but in the binary, the colors are actually
>>>>>> backwards, so color - FF AB CD EF becomes binary - EF CD AB FF, but that's
>>>>>> an easy change.
>>>>>> --CURRENTLY in my script, I'm doing this via a sub, passing an array
>>>>>> of the colors, the file, and the file location (directory), and pulling the
>>>>>> file, making the changes, and saving it back out via loop... BUT, because I
>>>>>> couldn't figure out how to just do the multiple replaces... I cheated and
>>>>>> made template files that contain "08 00 00 1C 11 11 11 11" and "08 00 00 1C
>>>>>> 22 22 22 22", so when I iterate through the loop, I just change like so:
>>>>>> for ( $i = 1 ; $i <= $COUNT ; $i++ ) {
>>>>>> ...
>>>>>>     $line =~ s|08\s00\s00\s1c\s$i$i\s$i$i\s$i$i\s$i$i\s|08 00 00 1c
>>>>>> $array_ref[$i]|;
>>>>>> ...
>>>>>> }
>>>>>>
>>>>>> Code works, but I'd much rather be able to pull native files straight
>>>>>> out of the .zip, and change them that way... :P
>>>>>>
>>>>>> Anyway, again, if there are specifics I missed, happy to provide them.
>>>>>>
>>>>>> I keep thinking there should be an easy way to do this... but my
>>>>>> google-fu is failing me..
>>>>>>
>>>>>> Thanks again for the help with the previous problem, and in advance
>>>>>> for any assistance on this one.  :)
>>>>>> At the very least, I guess I can provide some chatter to the group.
>>>>>>
>>>>>> -John (sephtin @gmail)
>>>>>>
>>>>>> _______________________________________________
>>>>>> Raleigh-talk mailing list
>>>>>> Raleigh-talk at pm.org
>>>>>> http://mail.pm.org/mailman/listinfo/raleigh-talk
>>>>>>
>>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> Raleigh-talk mailing list
>>>> Raleigh-talk at pm.org
>>>> http://mail.pm.org/mailman/listinfo/raleigh-talk
>>>>
>>>>
>>>
>>
>> _______________________________________________
>> Raleigh-talk mailing list
>> Raleigh-talk at pm.org
>> http://mail.pm.org/mailman/listinfo/raleigh-talk
>>
>>
>
> _______________________________________________
> Raleigh-talk mailing list
> Raleigh-talk at pm.org
> http://mail.pm.org/mailman/listinfo/raleigh-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.pm.org/pipermail/raleigh-talk/attachments/20110223/c8a272b4/attachment-0001.html>