Q:

Spark Streaming avoid check-point serialization issue when re-deploying the code

we have a spark streaming application and enabled check-pointing.
when we do code re-deployment, we need to delete check-point directory or give some other check-pointing directory, so there is going to be data loss, since we are using direct streaming approach.
is it something we can do with SerialVersionUID ? what will happen if we use the same SerialVersionUID when we do code re-deployment?

A:

is it something we can do with SerialVersionUID ? what will happen if we use the same SerialVersionUID when we do code re-deployment?

Don't do that. Since your code isn't binary compatible between releases, this isn't going to work, and in general you don't want to manually assign a serial version to every class in your project, that doesn't scale.
You will have to delete the checkpoint data, that is one of the main caveats of streaming. You can create additional bash scripts which makes sure to delete the checkpoint in case of a version upgrade.
Better yet, if you have a configuration management system you can directly integrate the update with deletion.

