From alex at owal.co.uk Wed Aug 22 01:45:32 2007 From: alex at owal.co.uk (alex at owal.co.uk) Date: Wed, 22 Aug 2007 09:45:32 +0100 (BST) Subject: [Banking-pm] Data::Reconciliation - possible work Message-ID: <2680.193.134.170.35.1187772332.squirrel@www.sflink.net> Does anyone here (or in London.pm) use Data::Reconciliation ? What I would like to do is to put it on a website which allows people to upload csv files, run them through Data::Reconciliation to get a report. (And yes - I might even pay!) I dont believe Data::Reconciliation is all that good - but it is all we have for now on CPAN which does the job. It is limited in terms of doing everything in memory which is a bit of a no-no. > http://search.cpan.org/author/CHATEAU/Data-Reconciliation-0.07/lib/Data/Reconciliation.pm Alex From billy at cowfish.org.uk Wed Aug 22 02:15:45 2007 From: billy at cowfish.org.uk (Billy Abbott) Date: Wed, 22 Aug 2007 10:15:45 +0100 (BST) Subject: [Banking-pm] What CPAN modules do you use? Message-ID: Prompted by Alex's mail just now - what finance/banking related CPAN modules, if any, do you use? I just did a quick search on cpan.org and couldn't find all that many, but am interested if anyone does use any. We don't use any as far as I know, going as far as having our own identifier checksum/checking modules, but am always on the lookout for things that might save effort. --billy -- Brain: Here we are, Pinky -- at the dawn of time! Pinky: Narf, Brain. Wake me at the noon of time. Billy Abbott billy at cowfish dot org dot uk From alex at owal.co.uk Wed Aug 22 02:47:45 2007 From: alex at owal.co.uk (alex at owal.co.uk) Date: Wed, 22 Aug 2007 10:47:45 +0100 (BST) Subject: [Banking-pm] What CPAN modules do you use? In-Reply-To: References: Message-ID: <60787.193.134.170.35.1187776065.squirrel@www.sflink.net> > Prompted by Alex's mail just now - what finance/banking related CPAN > modules, if any, do you use? Well the first thing to note is that I find the Sybase modules far more common in Banking than elsewhere. We use both the DBI one and a pre-DBI one. Also fairly common is anything to do with reading/writing CSV and Excel files. Here are some more random modules we might have installed and may or may not be used. Algorithm::CurveFit 1.02 Business::ISIN 0.20 (My favourite debugging tool) Data::Dumper 2.121_02 Date::Format 2.22 Date::Manip 5.44 Spreadsheet::WriteExcel 2.18 Text::CSV_XS 0.23 and loads of other normal stuff. Apart from the ISIN one (which probably isnt used) none of them are finance specific. I could post my Bundle::Snapshot but am not sure about confidentiality ;-) Alex From billy at cowfish.org.uk Wed Aug 22 02:56:27 2007 From: billy at cowfish.org.uk (Billy Abbott) Date: Wed, 22 Aug 2007 10:56:27 +0100 (BST) Subject: [Banking-pm] What CPAN modules do you use? In-Reply-To: <60787.193.134.170.35.1187776065.squirrel@www.sflink.net> References: <60787.193.134.170.35.1187776065.squirrel@www.sflink.net> Message-ID: On Wed, 22 Aug 2007 alex at owal.co.uk wrote: >> Prompted by Alex's mail just now - what finance/banking related CPAN >> modules, if any, do you use? > Algorithm::CurveFit 1.02 > Business::ISIN 0.20 > (My favourite debugging tool) > Data::Dumper 2.121_02 > > Date::Format 2.22 > Date::Manip 5.44 > > Spreadsheet::WriteExcel 2.18 > Text::CSV_XS 0.23 > > and loads of other normal stuff. I would consider almost all of those (apart from the first couple) as normal stuff :) > Apart from the ISIN one (which probably isnt used) none of them are > finance specific. Things like the ISIN module were what I was referring to - things which are more finance specific. There are a bunch of industry specific things up on CPAN for many different industries, I was just wondering if there are any that cater to our side of things that people actually use. I'm trying to think of new perl module ideas that I might put together and have started wavering towards hacking on some finance related perl in my spare time... --billy -- Transvestite ninjas - how did I not see that coming? Billy Abbott billy at cowfish dot org dot uk From mark at blackmans.org Wed Aug 22 05:18:12 2007 From: mark at blackmans.org (Mark Blackman) Date: Wed, 22 Aug 2007 13:18:12 +0100 Subject: [Banking-pm] What CPAN modules do you use? In-Reply-To: References: <60787.193.134.170.35.1187776065.squirrel@www.sflink.net> Message-ID: <0DB50E24-B966-455A-B013-949F46A07E5A@blackmans.org> On 22 Aug 2007, at 10:56, Billy Abbott wrote: > > Things like the ISIN module were what I was referring to - things > which > are more finance specific. There are a bunch of industry specific > things > up on CPAN for many different industries, I was just wondering if > there > are any that cater to our side of things that people actually use. Anyone using Math::BigInt or Math::BigFloat to deal with accumulation of FP errors in computations? As illustrated by the trivial example of perl -e 'print "accumulation error!\n" if (0.25+0.33 != 0.58)'; - Mark From a.r.ferreira at gmail.com Wed Aug 22 10:39:47 2007 From: a.r.ferreira at gmail.com (Adriano Ferreira) Date: Wed, 22 Aug 2007 14:39:47 -0300 Subject: [Banking-pm] What CPAN modules do you use? In-Reply-To: <0DB50E24-B966-455A-B013-949F46A07E5A@blackmans.org> References: <60787.193.134.170.35.1187776065.squirrel@www.sflink.net> <0DB50E24-B966-455A-B013-949F46A07E5A@blackmans.org> Message-ID: <73ddeb6c0708221039s357e52e7oded35faedef00eb5@mail.gmail.com> On 8/22/07, Mark Blackman wrote: > > On 22 Aug 2007, at 10:56, Billy Abbott wrote: > > > > Things like the ISIN module were what I was referring to - things > > which > > are more finance specific. There are a bunch of industry specific > > things > > up on CPAN for many different industries, I was just wondering if > > there > > are any that cater to our side of things that people actually use. > > Anyone using Math::BigInt or Math::BigFloat to deal with accumulation > of FP errors in computations? > > As illustrated by the trivial example of > > perl -e 'print "accumulation error!\n" if (0.25+0.33 != 0.58)'; If you're prepared for the hit performance, then: $ perl -Mbignum -e 'print "ok!\n" if (0.25+0.33 == 0.58)'; ok! works wonderfully. And don't forget these modules received a lot of attention recently. Many changes were introduced for the future 5.10 perl release and are available already in the CPAN distributions. > > - Mark > _______________________________________________ > Banking-pm mailing list > Banking-pm at pm.org > http://mail.pm.org/mailman/listinfo/banking-pm > From mark at blackmans.org Wed Aug 22 12:56:21 2007 From: mark at blackmans.org (Mark Blackman) Date: Wed, 22 Aug 2007 20:56:21 +0100 Subject: [Banking-pm] What CPAN modules do you use? In-Reply-To: <73ddeb6c0708221039s357e52e7oded35faedef00eb5@mail.gmail.com> References: <60787.193.134.170.35.1187776065.squirrel@www.sflink.net> <0DB50E24-B966-455A-B013-949F46A07E5A@blackmans.org> <73ddeb6c0708221039s357e52e7oded35faedef00eb5@mail.gmail.com> Message-ID: <9B420FE3-EAF6-4EE8-B4A3-2751E2006A8E@blackmans.org> On 22 Aug 2007, at 18:39, Adriano Ferreira wrote: > On 8/22/07, Mark Blackman wrote: >> >> On 22 Aug 2007, at 10:56, Billy Abbott wrote: >>> >>> Things like the ISIN module were what I was referring to - things >>> which >>> are more finance specific. There are a bunch of industry specific >>> things >>> up on CPAN for many different industries, I was just wondering if >>> there >>> are any that cater to our side of things that people actually use. >> >> Anyone using Math::BigInt or Math::BigFloat to deal with accumulation >> of FP errors in computations? >> >> As illustrated by the trivial example of >> >> perl -e 'print "accumulation error!\n" if (0.25+0.33 != 0.58)'; > > If you're prepared for the hit performance, then: > > $ perl -Mbignum -e 'print "ok!\n" if (0.25+0.33 == 0.58)'; > ok! > > works wonderfully. And don't forget these modules received a lot of > attention recently. Many changes were introduced for the future 5.10 > perl release and are available already in the CPAN distributions. Well, as bignum is just the transparent version of Math::BigFloat/ BigInt, in a roundabout way, I'm asking does anyone on the list ever have a requirement to avoid these accumulation errors and is the requirement sufficiently hard that they are obliged to live with the performance hit to use Math::BigFloat? Do you use it for production work? Regards, Mark > >> >> - Mark >> _______________________________________________ >> Banking-pm mailing list >> Banking-pm at pm.org >> http://mail.pm.org/mailman/listinfo/banking-pm >> From mike at wormers.net Wed Aug 22 23:19:49 2007 From: mike at wormers.net (mike at wormers.net) Date: Thu, 23 Aug 2007 07:19:49 +0100 (BST) Subject: [Banking-pm] What CPAN modules do you use? In-Reply-To: <9B420FE3-EAF6-4EE8-B4A3-2751E2006A8E@blackmans.org> References: <60787.193.134.170.35.1187776065.squirrel@www.sflink.net> <0DB50E24-B966-455A-B013-949F46A07E5A@blackmans.org> <73ddeb6c0708221039s357e52e7oded35faedef00eb5@mail.gmail.com> <9B420FE3-EAF6-4EE8-B4A3-2751E2006A8E@blackmans.org> Message-ID: <4193.141.228.106.136.1187849989.squirrel@mail.wormers.net> Why use classes like BigFloat etc? Does it suggests a level of accuracy beyond the model? i.e. 'double' should be sufficient? The extra 'precision' probably isn't getting you a more accurate figure (does depend on what exactly you are computing though I suppose). > > On 22 Aug 2007, at 18:39, Adriano Ferreira wrote: > >> On 8/22/07, Mark Blackman wrote: >>> >>> On 22 Aug 2007, at 10:56, Billy Abbott wrote: >>>> >>>> Things like the ISIN module were what I was referring to - things >>>> which >>>> are more finance specific. There are a bunch of industry specific >>>> things >>>> up on CPAN for many different industries, I was just wondering if >>>> there >>>> are any that cater to our side of things that people actually use. >>> >>> Anyone using Math::BigInt or Math::BigFloat to deal with accumulation >>> of FP errors in computations? >>> >>> As illustrated by the trivial example of >>> >>> perl -e 'print "accumulation error!\n" if (0.25+0.33 != 0.58)'; >> >> If you're prepared for the hit performance, then: >> >> $ perl -Mbignum -e 'print "ok!\n" if (0.25+0.33 == 0.58)'; >> ok! >> >> works wonderfully. And don't forget these modules received a lot of >> attention recently. Many changes were introduced for the future 5.10 >> perl release and are available already in the CPAN distributions. > > Well, as bignum is just the transparent version of Math::BigFloat/ > BigInt, > in a roundabout way, I'm asking does anyone on the list ever have a > requirement > to avoid these accumulation errors and is the requirement > sufficiently hard > that they are obliged to live with the performance hit to use > Math::BigFloat? > > Do you use it for production work? > > Regards, > Mark > >> >>> >>> - Mark >>> _______________________________________________ >>> Banking-pm mailing list >>> Banking-pm at pm.org >>> http://mail.pm.org/mailman/listinfo/banking-pm >>> > > _______________________________________________ > Banking-pm mailing list > Banking-pm at pm.org > http://mail.pm.org/mailman/listinfo/banking-pm > > From mark at blackmans.org Thu Aug 23 01:58:21 2007 From: mark at blackmans.org (Mark Blackman) Date: Thu, 23 Aug 2007 09:58:21 +0100 Subject: [Banking-pm] What CPAN modules do you use? In-Reply-To: <4193.141.228.106.136.1187849989.squirrel@mail.wormers.net> References: <60787.193.134.170.35.1187776065.squirrel@www.sflink.net> <0DB50E24-B966-455A-B013-949F46A07E5A@blackmans.org> <73ddeb6c0708221039s357e52e7oded35faedef00eb5@mail.gmail.com> <9B420FE3-EAF6-4EE8-B4A3-2751E2006A8E@blackmans.org> <4193.141.228.106.136.1187849989.squirrel@mail.wormers.net> Message-ID: On 23 Aug 2007, at 07:19, mike at wormers.net wrote: > Why use classes like BigFloat etc? Does it suggests a level > of accuracy beyond the model? i.e. 'double' should be sufficient? The > extra 'precision' probably isn't getting you a more accurate figure > (does > depend on what exactly you are computing though I suppose). Well, in the current 'double' model if you need to know if two sums agree (i.e. different calculations that should produce identical results) results, you can't guarantee that unless you make a point of rounding at key stages. Math::BigFloat (and bignum) free you from the requirement to worry about rounding in your application, with some performance penalty. In any case, what I'm asking is, does anyone here on this list need to care about this of agreement of two computations and what do they tend to do about it? It's come up for me and the only two strategies I identified were * pay attention to your computations and round (via sprintf) at obvious points * delegate the whole problem to Math::BigFloat (or similar) the 'Accuracy' section of http://en.wikipedia.org/wiki/Floating_point makes the point as well and the general problem that floating point arithmetic in the IEEE754 standard is neither associative or distributive, but you might very well assume that is. - Mark From ns7d-l3x0 at xemaps.com Thu Aug 23 02:52:27 2007 From: ns7d-l3x0 at xemaps.com (IvorW) Date: Thu, 23 Aug 2007 10:52:27 +0100 Subject: [Banking-pm] What CPAN modules do you use? In-Reply-To: References: <60787.193.134.170.35.1187776065.squirrel@www.sflink.net> <0DB50E24-B966-455A-B013-949F46A07E5A@blackmans.org> <73ddeb6c0708221039s357e52e7oded35faedef00eb5@mail.gmail.com> <9B420FE3-EAF6-4EE8-B4A3-2751E2006A8E@blackmans.org> <4193.141.228.106.136.1187849989.squirrel@mail.wormers.net> Message-ID: <46CD58DB.4030404@tiscali.co.uk> Mark Blackman wrote: > > On 23 Aug 2007, at 07:19, mike at wormers.net wrote: > > > > > >> >> Why use classes like BigFloat etc? Does it suggests a level >> >> of accuracy beyond the model? i.e. 'double' should be sufficient? The >> >> extra 'precision' probably isn't getting you a more accurate figure >> >> (does >> >> depend on what exactly you are computing though I suppose). >> >> >> > > > > Well, in the current 'double' model if you need to know if two > > sums agree (i.e. different calculations that should produce identical > > results) > > results, you can't guarantee that unless you make a point of rounding > > at key stages. Math::BigFloat (and bignum) free you from the requirement > > to worry about rounding in your application, with some performance > > penalty. > > > > In any case, what I'm asking is, does anyone here on this list > > need to care about this of agreement of two computations and what > > do they tend to do about it? It's come up for me and the only two > > strategies > > I identified were > > > > * pay attention to your computations and round (via sprintf) at > > obvious points > > * delegate the whole problem to Math::BigFloat (or similar) > > > > the 'Accuracy' section of http://en.wikipedia.org/wiki/Floating_point > > makes the point as well and the general problem that floating point > > arithmetic in the IEEE754 standard is neither associative or > > distributive, > > but you might very well assume that is. > > > When banks are trading swaps, to the value of trillions of pounds, notional, per annum, the rounding amounts could well become significant. Organisations such as the FSA and ISDA do have written rules for rounding, and it is up to the banks, brokers, exchanges and clearing houses to follow the rules. Some of my time at LCH was spent chasing rounding bugs. There were particular counterparty banks that were more fussy than others. LCH did use double precision float as their data type for storing monetary values, much to my chagrin. Ivor. From mike at wormers.net Thu Aug 23 03:01:55 2007 From: mike at wormers.net (mike at wormers.net) Date: Thu, 23 Aug 2007 11:01:55 +0100 (BST) Subject: [Banking-pm] What CPAN modules do you use? In-Reply-To: References: <60787.193.134.170.35.1187776065.squirrel@www.sflink.net> <0DB50E24-B966-455A-B013-949F46A07E5A@blackmans.org> <73ddeb6c0708221039s357e52e7oded35faedef00eb5@mail.gmail.com> <9B420FE3-EAF6-4EE8-B4A3-2751E2006A8E@blackmans.org> <4193.141.228.106.136.1187849989.squirrel@mail.wormers.net> Message-ID: <17069.141.228.106.136.1187863315.squirrel@mail.wormers.net> Just curious about your useage thats all... re: evaluating if two numbers are the same. if two real numbers are equal you won't find a piece of model code that says double x; double y; .... if ( x == y ) there will be (at the model level) some 'IsEqual' method that will default a precision. My point being that inside a model say its pointless to use greater than double b/c it implies a level of accurracy that simply doesn't exist. re: adding up swap notionals or some other total - then I'm suprised you'd need it.. > > On 23 Aug 2007, at 07:19, mike at wormers.net wrote: > >> Why use classes like BigFloat etc? Does it suggests a level >> of accuracy beyond the model? i.e. 'double' should be sufficient? The >> extra 'precision' probably isn't getting you a more accurate figure >> (does >> depend on what exactly you are computing though I suppose). > > Well, in the current 'double' model if you need to know if two > sums agree (i.e. different calculations that should produce identical > results) > results, you can't guarantee that unless you make a point of rounding > at key stages. Math::BigFloat (and bignum) free you from the requirement > to worry about rounding in your application, with some performance > penalty. > > In any case, what I'm asking is, does anyone here on this list > need to care about this of agreement of two computations and what > do they tend to do about it? It's come up for me and the only two > strategies > I identified were > > * pay attention to your computations and round (via sprintf) at > obvious points > * delegate the whole problem to Math::BigFloat (or similar) > > the 'Accuracy' section of http://en.wikipedia.org/wiki/Floating_point > makes the point as well and the general problem that floating point > arithmetic in the IEEE754 standard is neither associative or > distributive, > but you might very well assume that is. > > - Mark > > > > From mark at blackmans.org Thu Aug 23 03:35:18 2007 From: mark at blackmans.org (Mark Blackman) Date: Thu, 23 Aug 2007 11:35:18 +0100 Subject: [Banking-pm] What CPAN modules do you use? In-Reply-To: <17069.141.228.106.136.1187863315.squirrel@mail.wormers.net> References: <60787.193.134.170.35.1187776065.squirrel@www.sflink.net> <0DB50E24-B966-455A-B013-949F46A07E5A@blackmans.org> <73ddeb6c0708221039s357e52e7oded35faedef00eb5@mail.gmail.com> <9B420FE3-EAF6-4EE8-B4A3-2751E2006A8E@blackmans.org> <4193.141.228.106.136.1187849989.squirrel@mail.wormers.net> <17069.141.228.106.136.1187863315.squirrel@mail.wormers.net> Message-ID: <09F7D699-91A7-4995-A591-42213A24791F@blackmans.org> On 23 Aug 2007, at 11:01, mike at wormers.net wrote: > Just curious about your useage thats all... > > re: evaluating if two numbers are the same. > > if two real numbers are equal you won't find a piece of model code > that says > double x; > double y; > .... > if ( x == y ) > > there will be (at the model level) some 'IsEqual' method that will > default > a precision. My point being that inside a model say its pointless > to use > greater than double b/c it implies a level of accurracy that simply > doesn't exist. > > re: adding up swap notionals or some other total - then I'm > suprised you'd > need it.. Ok, in my case it was a balance comparison and it was actually a >= test that failed when it shouldn't have done. It sounds like you're suggesting that you abandon (or perhaps overload) the built-in operators when working with real numbers, which certainly makes sense, given the limitations of IEEE754. - Mark From alex at owal.co.uk Thu Aug 30 05:27:14 2007 From: alex at owal.co.uk (alex at owal.co.uk) Date: Thu, 30 Aug 2007 13:27:14 +0100 (BST) Subject: [Banking-pm] [Fwd: [ANNOUNCE] [SOCIAL] Social at Dirty Dicks, Thursday 6th September] Message-ID: <58874.193.134.170.35.1188476834.squirrel@www.sflink.net> Since the London.pm social is happening "in the City" next week, perhaps we should try to make it there! Alex ---------------------------- Original Message ---------------------------- Subject: [ANNOUNCE] [SOCIAL] Social at Dirty Dicks, Thursday 6th September From: "Simon Wistow" Date: Thu, August 30, 2007 11:55 am To: "London.pm Announce" -------------------------------------------------------------------------- Because we're cool, outre, avant garde kind of people and we like to try new, zany things we're trying yet another new pub this month - the delightfully named Dirty Dicks by Liverpool Street. And when I say "by Liverpool Street" I really mean it - it's opposite the Bishopsgate Entrance to Liverpool Street Station. http://london.pm.org/meetings/locations/dirty_dicks.html http://www.dirtydicks.co.uk/ http://london.randomness.org.uk/wiki.cgi?Dirty_Dick's,_EC2M_4NR http://streetmap.co.uk/newmap.srf?x=533359&y=181631 It's a Young's Pub which means tasty, tasty beer (the last time I was there I seem to remember they had Double Chocolate on tap - rarr!) Social meets are a chance for the various members of the group to meet up face to face and chat with each other about things - both Perl and non Perl - and newcomers are more than welcome The monthly meets tend to be bigger than the other ad hoc meetings that take place at other times and we make sure that they're in easy to get to locations and the pub serves food (meaning that people can eat in the bar if they want to.) They normally start around 6.30pm (or whenever people get there after work) and a group tends to be left come closing time. If you're a new comer or other first timer (even if you've been lurking on the mailing list or on IRC) then please seek me out - we have a tradition that the leader of this motley crew buys the new people a drink (alcoholic or not, either's fine) and introduce them to people. Simon Location Opposite Liverpool Street station. By Tube Liverpool Street Tube couldn't be much closer. If you're feeling perverse then Aldgate, Mansion House, Moorgate and Shoreditch are all within walking distance. By Bus Oh, so many buses - check them out from the TFL map http://www.tfl.gov.uk/tfl/gettingaround/maps/buses/pdf/liverpoolstreet-2161.pdf