Connect miners to central server/wallet

It seems that Primecoin HP is the miner for solo mining… what if I have a minning rig and want to connect a bunch of remote miners to my single full blown Internet connected client (with my wallet and such)? Is it possible to run primecoind as miner only, and connect each miner via RPC to my full client?

Short answer: Not possible so far.