Please use this identifier to cite or link to this item:
|Title: ||HiSeqGAN: 高維資料的序列合成與預測|
HiSeqGAN: High-dimensional Sequence Synthesis and Prediction
|Issue Date: ||2019-08-07 16:05:57 (UTC+8)|
High-dimensional data sequences constantly appear in practice. State-of-the-art models such as recurrent neural networks suffer prediction accuracy from complex relations among values of attributes. Adopting unsupervised clustering that clusters data based on their attribute value similarity results data in lower dimensions that can be structured in a hierarchical relation. It is essential to consider these data relations to improve the performance of training models. In this work, we propose a new approach to synthesize and predict sequences of data that are structured in a hierarchy. Specifically, we adopt a new hierarchical data encoding and seamlessly modify loss functions of SeqGAN as our training model to synthesize data sequences. In practice, we first use the hierarchical clustering algorithm, GHSOM, to cluster our training data. By relabelling a sample with the cluster that it falls to, we are able to use the GHSOM map to identify the hierarchical relation of samples. We then converse the clusters to the coordinate vectors with our hierarchical data encoding algorithm and replace the loss function with maximizing cosine similarity in the SeqGAN model to synthesize cluster sequences. Using the synthesized sequences, we are able to achieve better performance on high-dimension data training and prediction compared to the state-of-the-art models.
|Reference: ||*  P. S. Churchland, T. J. Sejnowski, and T. A. Poggio, The computational brain. MIT press, 2016.|
*  W. Awad and S. ELseuofi, “Machine learning methods for e-mail classification,” International Journal of Computer Applications, vol. 16, no. 1, 2011.
*  F. Sebastiani, “Machine learning in automated text categorization,” ACM computing surveys (CSUR), vol. 34, no. 1, pp. 1–47, 2002.
*  W. B. Rauch-Hindin, Artificial Intelligence in Business, Science, and Industry: Fun- damentals. Prentice-Hall New Jersey, 1986.
*  K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556, 2014.
*  Y. LeCun, Y. Bengio et al., “Convolutional networks for images, speech, and time series,” The handbook of brain theory and neural networks, vol. 3361, no. 10, p. 1995, 1995.
*  R. J. Williams and D. Zipser, “A learning algorithm for continually running fully recurrent neural networks,” Neural computation, vol. 1, no. 2, pp. 270–280, 1989.
*  I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in neural information processing systems, 2014, pp. 2672–2680.
*  O. Abdel-Hamid, A.-r. Mohamed, H. Jiang, L. Deng, G. Penn, and D. Yu, “Convolu- tional neural networks for speech recognition,” IEEE/ACM Transactions on audio, speech, and language processing, vol. 22, no. 10, pp. 1533–1545, 2014.
*  I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” in Advances in neural information processing systems, 2014, pp. 3104– 3112.
*  A. Graves, A.-r. Mohamed, and G. Hinton, “Speech recognition with deep recur- rent neural networks,” in Acoustics, speech and signal processing (icassp), 2013 ieee international conference on. IEEE, 2013, pp. 6645–6649.
*  S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997.
*  F. A. Gers, J. Schmidhuber, and F. Cummins, Learning to forget: Continual predic- tion with LSTM. IET, 1999.
*  L. Yu, W. Zhang, J. Wang, and Y. Yu, “Seqgan: Sequence generative adversarial nets with policy gradient.” in AAAI, 2017, pp. 2852–2858.
*  A. Creswell and A. A. Bharath, “Denoising adversarial autoencoders,” IEEE trans- actions on neural networks and learning systems, no. 99, pp. 1–17, 2018.
*  A. Makhzani, J. Shlens, N. Jaitly, I. Goodfellow, and B. Frey, “Adversarial autoen- coders,” arXiv preprint arXiv:1511.05644, 2015.
*  A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learn- ing with deep convolutional generative adversarial networks,” arXiv preprint arXiv:1511.06434, 2015.
*  T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, “Im- proved techniques for training gans,” in Advances in Neural Information Processing Systems, 2016, pp. 2234–2242.
*  P. Vincent, H. Larochelle, Y. Bengio, and P.-A. Manzagol, “Extracting and compos- ing robust features with denoising autoencoders,” in Proceedings of the 25th inter- national conference on Machine learning. ACM, 2008, pp. 1096–1103.
*  P. B. Myszkowski and B. Buczek, “Growing hierarchical self-organizing map for searching documents using visual content,” in 2011 Federated Conference on Com- puter Science and Information Systems (FedCSIS). IEEE, 2011, pp. 77–81.
*  J. Dai, Y. Lu, and Y.-N. Wu, “Generative modeling of convolutional neural net- works,” arXiv preprint arXiv:1412.6296, 2014.
*  J. Xie, Y. Lu, S.-C. Zhu, and Y. Wu, “A theory of generative convnet,” in Interna- tional Conference on Machine Learning, 2016, pp. 2635–2644.
*  J. Xie, Y. Lu, R. Gao, S.-C. Zhu, and Y. N. Wu, “Cooperative training of descriptor and generator networks,” arXiv preprint arXiv:1609.09408, 2016.
*  Z. Dai, Z. Yang, F. Yang, W. W. Cohen, and R. R. Salakhutdinov, “Good semi- supervised learning that requires a bad gan,” in Advances in Neural Information Processing Systems, 2017, pp. 6510–6520.
*  E. L. Denton, S. Chintala, R. Fergus et al., “Deep generative image models using a laplacian pyramid of adversarial networks,” in Advances in neural information processing systems, 2015, pp. 1486–1494.
*  J. Ho and S. Ermon, “Generative adversarial imitation learning,” in Advances in Neural Information Processing Systems, 2016, pp. 4565–4573.
*  P. Luc, C. Couprie, S. Chintala, and J. Verbeek, “Semantic segmentation using adversarial networks,” arXiv preprint arXiv:1611.08408, 2016.
*  M.-Y. Liu and O. Tuzel, “Coupled generative adversarial networks,” in Advances in neural information processing systems, 2016, pp. 469–477.
*  J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” arXiv preprint, 2017.
*  S. Tulyakov, M.-Y. Liu, X. Yang, and J. Kautz, “Mocogan: Decomposing motion and content for video generation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1526–1535.
*  M. Saito, E. Matsumoto, and S. Saito, “Temporal generative adversarial nets with singular value clipping,” in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2830–2839.
*  C. Vondrick, H. Pirsiavash, and A. Torralba, “Generating videos with scene dynam- ics,” in Advances In Neural Information Processing Systems, 2016, pp. 613–621.
*  J. Xie, R. Gao, Z. Zheng, S.-C. Zhu, and Y. N. Wu, “Learning dynamic generator model by alternating back-propagation through time,” arXiv preprint arXiv:1812.10587, 2018.
*  A. Hadriche, N. Jmail, and R. Elleuch, “Different methods of partitioning the phase space of a dynamic system,” International Journal of Computer Applications, vol. 93, pp. 1–5, 05 2014.
*  M. Dittenbach, D. Merkl, and A. Rauber, “The growing hierarchical self-organizing map,” in Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium, vol. 6. IEEE, 2000, pp. 15–19.
*  J. Macqueen, “Some methods for classification and analysis of multivariate observa- tions,” in In 5-th Berkeley Symposium on Mathematical Statistics and Probability, 1967, pp. 281–297.
*  T. Kohonen, Neurocomputing: Foundations of Research, J. A. Anderson and E. Rosenfeld, Eds. Cambridge, MA, USA: MIT Press, 1988. [Online]. Available: http://dl.acm.org/citation.cfm?id=65669.104428
*  M. D. E. P. Andreas Rauber, Dieter Merkl, “The growing hierarchical self-organizing map.” [Online]. Available: http://www.ifs.tuwien.ac.at/∼andi/ghsom/
*  V. Rajagopalan, A. Ray, R. Samsi, and J. Mayer, “Pattern identification in dynamical systems via symbolic time series analysis,” Pattern Recognition, vol. 40, no. 11, pp. 2897–2907, 2007.
*  T. Y. Lin, H. H. C. Chuang, and F. Yu, “Tracking supply chain process variabil- ity with unsupervised cluster traversal,” in 2018 IEEE 16th Intl Conf on Depend- able, Autonomic and Secure Computing, 16th Intl Conf on Pervasive Intelligence and Computing, 4th Intl Conf on Big Data Intelligence and Computing and Cyber Science and Technology Congress (DASC/PiCom/DataCom/CyberSciTech). IEEE, 2018, pp. 966–973.
*  J. R. Quinlan et al., “Bagging, boosting, and c4. 5,” in AAAI/IAAI, Vol. 1, 1996, pp. 725–730.
*  J. Demˇsar, T. Curk, A. Erjavec, Cˇrt Gorup, T. Hoˇcevar, M. Milutinoviˇc, M. Moˇzina, M. Polajnar, M. Toplak, A. Stariˇc, M. Sˇtajdohar, L. Umek, L. Zˇagar, J. Zˇbontar, M. Zˇitnik, and B. Zupan, “Orange: Data mining toolbox in python,” Journal of Machine Learning Research, vol. 14, pp. 2349–2353, 2013. [Online]. Available: http://jmlr.org/papers/v14/demsar13a.html
|Source URI: ||http://thesis.lib.nccu.edu.tw/record/#G0106356004|
|Data Type: ||thesis|
|Appears in Collections:||[資訊管理學系] 學位論文|
Files in This Item:
All items in 政大典藏 are protected by copyright, with all rights reserved.