They are most likely not the source of the lag as of now; The amounts are still fairly tame (Maybe around 1000 crops total), but will fairly quickly rise as people start looking into them. I personally noticed a 14ms decrease in ticktime on my private server after unloading 3500 crops + the machinery for processing the produce, which I am aware is not proper profiling.
For the alternatives:
Making chunkloading more restrictive would hurt the pack in other ways more than it would help here. Most people already have vertical farms, so either you cannot chunkload that at all, causing problems with loading miners, among other things, or you can chunkload a very limited amount, which wouldn't really matter for the farms, as you only need like 2x2 for a decent sized farm.
Making crops grow faster (or increasing yield) would also be a decent option, even if it made the process of getting the seeds easier, although the hard part is achieving a seed, not duplicating it. Not sure how to implement this, though.
Threading the crops would be an ideal solution, if that's possible at all.