The speed will decrease during periods you'd think there would be heavy usage (mid morning/late afternoon, even slower in early evening). Then speed up dramatically in the late evening when most have gone to bed.
fwiw I see the same congestion-related throughput pattern on my copper cableco connection (fast.com reports much lower numbers in the evening and on the weekend vs early in the morning on a weekday). Typically I can still get full speed from my own servers though, implying the congestion is specific to the peering between my ISP and Netflix CDN nodes.
