More and more new services are emerging in the Internet, and these new services are accompanied with different quality of service (QoS) requirements which have brought new questions to network administration and management. While new applications need service guarantees from the network, the traditional applications function well without any level of service. Thus, functionality should be designed into the Internet to provide bandwidth sharing and prioritization. We believe that traffic classification and traffic modeling are helpful to provide QoS in IP networks. In this thesis we propose a knowledge-based traffic classification and modeling system based on combination of unsupervised learning algorithms with supervised learning algorithm, and model libraries. The system is knowledge-based in that we not only take advantage of the real time measurements from the online observed traffic, but also collect the historical information from past traffic traces. The scheme can be viewed as a two-step procedure: in the first step, we collect traffic traces from the studied network and examine the header information from each packet. Classification rules are developed based on header information, training samples and the use of unsupervised and supervised learning algorithms. In the second step, the classification rules are used to classify the active traffic into several QoS classes, then a best-fit statistical model is chosen from the model library to represent the traffic in each class. The work is divided into two parts. In the first part, we investigate a two-model library consisting of a short-range dependent (SRD) model and a long-range dependent (LRD) model. In order to study the model library, the models that are used in our work are presented and studied, the estimator of a statistic, Hurst parameter, are described. The parameters that are used in the estimator, AV wavelet-based estimator, are searched for suitable values. We experimentally evaluate the accuracy of Bayes Minimum Error (BME) classifier for static traffic, and its capability to catch traffic changes. Results show that the BME classifier can be quite accurate, and can catch changes in the traffic with a sliding window. In the second part, we study how to classify the traffic based on some packet header information. Four traffic traces are studied based on flow and connection analysis. We first perform data clustering using unsupervised algorithms, then we define QoS classes based on the natural clusters and choose four applications as the training data sets. From experiments, we determine the combination of feature statistics that are unique to the representative applications. A supervised learning algorithm, k nearest neighbor (KNN) algorithm, was used for classification. Leave-one-out cross-validation results show that the error rate of this algorithm was between 1.7% to 3.0% for k