snousle: (cigar)
[personal profile] snousle
Busy week. After the wedding, it was time to dive into data analysis again. Got in a good solid week.

Then I had a dinner for a group of Japanese wine buyers at Parducci. Just nine people, and a tight schedule. Aside from an antipasto plate, I served a cold asparagus salad with grilled eggplant, sun dried tomatoes, and tarragon mayonnaise, the latter being wonderful due to the fantastic eggs I've been getting lately. Then it was seared salmon and prawns with pappardelle and fava beans. I'd set up a camp chef range just outside the tiny kitchen so as to deal with the smoke, and the drama of it played really well with the guests. Finally, strawberry shortcake with candy cap ice cream. Unfortunately there were some technical difficulties with the ice cream machine and it came out more like a milkshake. Sigh... It was stunningly delicious, but the dessert as a whole fell very short of what it might have been.

An appreciative group, but my work continues to be plagued by small technical problems. Aside from the ice cream, the timing problems associated with fresh pasta are constantly messing me up. I need to figure out how to hold these dishes for fifteen minutes or so without them turning to rubber. And resist the urge to make too much - just two ounces a person is plenty, particularly when I make it using all yolks.

If that weren't enough, I'm now sitting on an airplane that's about to depart for Miami, where I'll be spending the week at Envoy. Got a nice relational database set up for their data, so it's time for some face to face chitchat to figure out where the analysis strategy is headed. My heavens, scientific computing is easy these days, especially with 64 bit processors. I used to spend virtually all my time dealing with resource limitations and data bottlenecks. Now that such concerns are nearly a thing of the past, i have to spend all my time thinking! So in a way, while specific problems are much easier, the work as a whole is "harder" because the mindless, tedious stuff has been taken away, leaving the hard parts to consume all the time!

I'm SO tired, this is perhaps the first time I've looked forward to flying. At last, some rest...

Time to turn off electronic devices, catch ya all later...

Date: 2010-06-15 06:48 am (UTC)
From: [identity profile] oscarlikesbugsy.livejournal.com
I love data diving. In fact, there was a time when I did it for a living. Even traveled to hear some prof in Santa Monica go on about his new data visualization techniques.

Anyway, I'm curious, what's the standard for a good relational database, these days? Do you set one up inside the analyitc software, or are there decent stand alone db progs that are easy enough for user manipulation/access?

(Sorry to hear about the milkshake ice-cream, but it still sounds delicious, like a snow ice-cream).

Date: 2010-06-15 06:22 pm (UTC)
From: [identity profile] snousle.livejournal.com
I've been quite happy with Microsoft sqlserver. At one time, big data meant Oracle, but sqlserver has definitely caught up and is much easier to manage. Hundreds of millions of rows is not a big deal anymore. Embedded databases are generally not suitable for large scale data management, and many analytic tools talk easily to databases through odbc or sqlserver specific connections, so standalone is almost always the way to go.

MySql is popular with academics but IMHO it's not industrial-strength yet.

Date: 2010-06-20 08:26 pm (UTC)
From: [identity profile] oscarlikesbugsy.livejournal.com
Thanks. Good to know.

I have a copy of S-PLUS on the shelf, which I am in the process of dusting off. At one time, they were boasting their ability to manage large, "high frequency" datasets better than alternatives.

It is a great thing what has happened to desktop computing. On this machine, the most winsome chance I took was on on-board RAID (and by "chance" I mean, thing I hadn't tried but "paid up for"). It really makes whipping around large chunks much, much easier. So far, the RAID volume has been quite robust, despite using drives that aren't particularly suited to it.

Date: 2010-06-20 10:11 pm (UTC)
From: [identity profile] snousle.livejournal.com
You realize S+ is a proprietary version of R, which is free, right? R is very popular for microarray analysis but hardly anyone I know of uses S+.

Date: 2010-06-21 05:04 pm (UTC)
From: [identity profile] oscarlikesbugsy.livejournal.com
I did not know that. Thank you. Indeed, I didn't even know about the "R" project.

Perhaps I can show up at their conference in Maryland next month (not too far) and find out who uses it.

Profile

snousle: (Default)
snousle

August 2013

S M T W T F S
    123
45 678910
11121314151617
1819202122 2324
25262728293031

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Feb. 9th, 2026 01:41 pm
Powered by Dreamwidth Studios