Abstract: I consider the problem of semiparametric model building for linear regression models with potentially time-varying coefficients. By allowing the response variable and explanatory variables be jointly a nonstationary process, the proposed methods are widely applicable to nonstationary and dependent observations, for example time-varying autoregressive processes with heteroscedastic errors. We propose a local linear shrinkage method that is capable of achieving variable selection and parameter estimation simultaneously in a computationally efficient manner. Its selection consistency along with the favorable oracle property is established. Due to the fear of losing efficiency, an information criterion is further proposed for distinguishing time-varying and time-invariant components. Numerical examples are presented to illustrate the proposed methods.