It's possible to use "autoscript" files to provide uboot with instructions on what to boot (from) but those files do not exist in current 3.14 kernel images and they are compiled scripts (not just plain text files) so there's a little bit of uncharted work involved. If you don't provide instructions uboot will boot from external media if present.
If the slot is "loose" the easier cure is probably to open the case and ensure the connections to the board are soldered properly and the connector tabs that locate the connector housing in the PCB are either tensioned correctly. Or maybe use a hot glue gun to get things under control.