In this paper, we consider Bayesian linear models for large datasets. We discuss two distinct strategies for generating Bayesian linear models with a large number of observations. The first approach employs an efficient Markov chain Monte Carlo (MCMC) method, while the second approach is exact and is based on a modification of Matheron's update rule (MUR) using Bayes' rule. We prove that MUR can be adapted for a large number of observations, resulting in a significant reduction in computational cost. The main advantage of these approaches is that sampling is performed before conditioning rather than after. This allows for the use of highly efficient samplers to generate the prior Gaussian vector when the precision covariance matrix exhibits special structures, such as Toeplitz, block-Toeplitz or sparsity. An empirical comparison between these two efficient approaches in terms of computational running time and prediction accuracy is conducted using both synthetic and real-world data studies.