You can restrict the linked records options by adding a "conditional filter" to your field. When a filter is selected on the form, it will limit the linked records available in the list. The conditional filters for the form can only use the linked records field and lookup fields. To elaborate on this functionality, you can watch this video.

You can hide this filter from the field and use prefill to fill it up. You just need to set the Conditional linked record filtering fields type with the Hide filter fields (prefill using URL) option.