BYO CNC Lathe ?

Ok,
I've taken an interest in purchasing a small lathe to convert to CNC.
Everything I look at though, spooks me off a bit. The Taig...To Small.
The Sherline, To Pricey. The Imports ? Well, we picked up a Jet 9x20 for
a quick and dirty machine and I was not very impressed. I did not think
one should build a machine like that out of such soft metal. You should
see how wobbled out the wrench holes are in the spindle from taking the
chuck off ! Scary! I do not think I would ever cnc that thing.

So, I began to think about building my own. (BYO)
Is there any reason why I could not use some good sized THK-IKO style
bearings for the rails ? Why do all lathes have the typical beds with
sliding gib concepts anyhow ? Linear bearings should work, No ?

I could even put the Tailstock on the same rails with a separate
leadscrew and handwheel instead of having the tailstock made in the
conventional way. That sounds alright.

I could come up with a really neat spindle with a really large thru hole
instead of all this too small stuff. (I knew I would someday find a use
for front wheel drive cars... The front hub and spindle ! I could keep
my GM lathe ALL GM !)

I thought I could probably use big chunks of Billet aluminum for the
headstock and tailstock but was worried about not getting enough mass.
But I suppose it would still be as good as a Taig or Sherline.

Thoughts ? Am I nuts ?

Chris L