Sea ice concentration estimation is crucial for secure ship navigation and ice hazard forecasting. In this paper, we propose a Convolutional Neural Network (CNN) architecture for sea ice concentration estimation over the Baltic Sea using two imaging modalities: Sentinel-1 and advanced microwave scanning radiometer2 (AMSR2). The main idea for fusing these two sensors is Sentinel-1 images have high spatial resolution and AMSR2 provides images independent of wind conditions. Our two-stream architecture is to preserve all possible information of the different resolution inputs, instead of interpolating the inputs to the same resolution while losing potentially useful information. We also investigate the impact of two loss functions and skip connection on the performance of the proposed CNN model. The experimental results show that CNN with focal loss function and skip connection can achieve R2 score of 90.6%.