New optical solitons and modulation instability analysis of generalized coupled nonlinear Schrödinger–KdV system

In this study, the generalized coupled nonlinear Schrödinger–KdV (NLS–KdV) system is investigated to obtain new optical soliton solutions. This system appears as a model for reciprocity between long and short waves in various physical settings. Different kinds of new soliton solutions including dark, bright, combined dark-bright, singular and combined singular soliton solutions are obtained using two effective methods namely, the extended sinh–Gordon equation expansion method and the solitary wave ansatz method. In addition, the modulation instability analysis of the system is presented based on the standard linear-stability analysis. The behaviours of obtained solutions are expressed by 3D graphs.