Apparently performance on larger context kinda sucks. Still impressive we have such large context on open source model.