The basic difference between the two is that RedirectPermanent sends the browser an HTTP 301(Moved Permanently) status code whereas Redirect will send an HTTP 302 status code.
Use RedirectPermanent if the resource has been moved permanently and will no longer be accessible in its previous location. Most browsers will cache this response and perform the redirect automatically without requesting the original resource again.
Use Redirect if the resource may be available in the same location (URL) in the future.
Example
Let's say that you have users in your system. You also have an option to delete existing users. Your website has a resource /user/{userid} that displays the details of a given user. If the user has been deleted, you must redirect to the /user/does-not-exist page. In this case:
If the user will never be restored again, you should use RedirectPermanent so the browser can go directly to /user/does-not-exist in subsequent requests even if the URL points to /user/{userid}.
If the user may be un-deleted sometime in the future, you should use a regular Redirect.
RedirectPermanent can be annoying while debugging because the second time you click, controller's action will be skipped and no break point are reached in the original function if it redirect elsewhere. That is unless you empty your cache before every click.