How do I mount a lathe chuck to a rotary table?

Hello, I just picked up a Grizzly 6 inch rotary table to add to my
collection of mini mill and mini lathe stuff. I wondered how I can
mount a 3 inch or 4 inch lathe chuck to my rotary table so I can use
it in the mill. It has T nut slots and an MT2 taper in the center.
I was going to make an adapter plate to do this but is there an
easier way?

Thanks alot,
Jamie