capitalist on "RE: Is Telecommuting Working?"
Two points.
1) All of these objections can be summed up with the statement that telecommuting requires competent managers. Since most managers are incompetent, this is clearly a problem.
2) The article fails to mention that the decision on whether to allow telecommuting is fundamentally political. Many managers and executives consider maintaining control of their employees more important than the success of the business. Many of them enter management because they enjoy controlling others; business efficiency is secondary, if it's even a consideration at all.
Publication date: 2008-08-22
More news from 2008-08-22
Dumber is Faster with Large Data Sets (and Disk Seeks)
I remember
reading Disk
is the new Tape earlier this year and how much it resonated.
That's probably because I was working for Yahoo at the time and
hearing a lot about their use
of Hadoop for data
processing. In fact, I even did a couple videos
(1
and 2)
about that.
Anyway, I recently faced the reality of this myself. When I wrote
about The
Long Term Performance of InnoDB I'd been beating my head against
a wall trying to get millions of records out of InnoDB
efficiently.
It was taking days to get all the records. Yes, days!
After joking that it'd probably be faster to just dump the tables
out and do the work myself in Perl, I thought about Disk is the
new Tape and realized what I was doing wrong.
Allow me to offer some background and explain...
There are several tables involved in the queries I needed to run.
Two of them are "core" tables and the other two are LEFT JOINed
because they hold optional data for the rows I'm pulling. There are
well over a hundred million records to consider and I need only
about 10-15% of them.
And these records fall into roughly 500 categories. So what I'd
been doing is fetching a list of categories, running a query for
each category to find the rows I actually need, processing the
results, and writing them to disk for further processing.
The query looked something like this:
SELECT field1, field2, field3, ... field N
FROM stuff_meta sm, stuff s
LEFT JOIN stuff_attributes sa ON sm.item_id = sa.item_id
LEFT JOIN stuff_dates sd ON sm.item_id = sd.item_id
WHERE sm.item_id = s.item_id
AND sm.cat_id = ?
AND sm.status IN ('A', 'B', 'C')
That seemed, at least in theory, to be the obvious way to
approach the problem. But the idea of waiting several days for the
results let me me to think a bit more about it (and to try some
InnoDB tuning along the way).
While it seems very counter-intuitive, this was sticking in my
head:
I?m still trying to get my head around this concept of
"linear" data processing. But I have found that I can do some
things faster by reading sequentially through a batch of files
rather than trying to stuff everything in a database (RDF or SQL)
and doing big join queries.
So I gave it a try. I wrote a new version of the code that
eliminated the two AND bits in the WHERE clause. Combining that
with
using mysql_use_result
in the client API, meant it had to process a stream of many tens of
millions of records, handle the status filtering and shorting
records into buckets based on cat_id (and some extra
bookkeeping).
As an aside, I should note that there used to be an ORDER BY on
that original query, but I abandoned that early on when I saw how
much work MySQL was doing to sort the records. While it made my
code a bit easier, it was far more efficient to track things outside
the database.
Anyway, the end result was that I was able to get all the data I
needed in merely 8 hours. In other words, treating MySQL as an SQL
powered tape drive yielded a 12 fold improvement in
performance.
Put another way, taking the brain-dead stupid, non-SQL,
mainframe-like approach got me results 12 times faster than doing it
the seemingly "correct" way.
Now this isn't exactly what the whole disk vs. tape thing
is about but it's pretty close. I'm aware that InnoDB works with
pages (that will contain multiple records, some of which I don't
need) and that's part of the problem in this scenario. But it's a
really interesting data point. And it's certainly going to change
my thinking about working with our data in the future.
Actually, it already has. :-)
Dumber is faster.
As I've
mentioned before, craigslist is hiring. We like good Perl/MySQL
hackers. And good sysadmin and network types too.
Ping me if you're
interested in either role.
(comments)
Publication date: 2008-08-22
more
capitalist on "RE: Is Telecommuting Working?"
Two points.
1) All of these objections can be summed up with the statement that telecommuting requires competent managers. Since most managers are incompetent, this is clearly a problem.
2) The article fails to mention that the decision on whether to allow telecommuting is fundamentally political. Many managers and executives consider maintaining control of their employees more important than the success of the business. Many of them enter management because they enjoy controlling others; business efficiency is secondary, if it's even a consideration at all.
Publication date: 2008-08-22
more
rebelegg on "Computer keep rebooting itself when playing games, movies"
This is my first time to use this forum so I posted the same topic on "Chip". Sry about that.
Im getting really tried on trying to fix my computer. I have tried to fix it for few months but never success.
Here is my computer, I built it few months ago and it should be a pretty good one so i really wanna fix it. THX:
CPU: Intel QX9650 Extreme 3.00GHz
MB: ASUS P5N-D 750i SLI motherboard
RAM: I tried two sets of RAM
1) Corsair CM2X1024-8500C5 (2*1G)
2) Kingston KVR800D2N5K2 (also 2*1G)
Video cards: I have dual graphic cards: NVIDIA Geforce 8800GTX in SLI mode
HDD: WD 500G
PSU: Antec 1000W power supply
OS: Win XP SP3
The computer reboot almost everytime I play game(Crysis, Need for speed prostreet, kungfu Panda, etc.. and even Age of Empire III now). It also reboot itself when i played movies sometime(not always though). I have turned off auto restarting of windows. But when it reboot, it doesn't give me any error message or blue screen. It just reboot, and go back to windows without causing any further problem.
I tried using only one video card and only one memory, reboot still. The first pairs of memory are not tested by ASUS so i brought the second pairs, which are listed on the motherboard user manual, but it still reboot.
It seems there is a little difference between using one video card and two. When using two video cards, it rebooted even i was playing some old games. when using only one video card and one RAM, it looks running little better but it still reboot once in a while.
I am thinking it's caused by PSU. But my PSU is 1000W, should be enough. And one thing that i almost forget to metion is, it didn't reboot when i run Prime95 for three hours.
I have used RealTemp, CoreTemp toreport the temp. The temp of the CPU (around 50C), M/B (40 something) and video card (60C - 70C something) seems normal. And i have six case fans but they are connected to a separate 110V AC-to-12V DC power adapte. So they are not using the power from PSU.
And i just reinstalled Win XP and the problem is still here. So I dont think it caused by virus.
To me it seems this can be caused by bad PSU, bad MB, RAM conflic, bad Geforce/nForce driver, or even SATA conflict. But i have no idea what to do and how to start to test it. I have spent so much money on this computer but it never makes me happy.
by the way, it doesn't reboot when i was using internet, MS word or playing music. But it does reboot when i burn DVD or making my own DVD-movie. So it seems it reboot when i running something related to graphic stuff.
Anyone have any idea? Many thanks!!
Lin
Publication date: 2008-08-22
more
Site generated in 0.536sec.