Sure:

$(function() {
   $('#flash').delay(500).fadeIn('normal', function() {
      $(this).delay(2500).fadeOut();
   });
});

jsFiddle example

This is a modification of Jacob's answer above. You can't fade in something that isn't hidden initially.

Instructions:

put an id of flash into your flash message, like this (my flash messages are stored here app/views/layouts/_flashmessages.html.erb):

<% flash.each do |key, value| %>
    <div class="well lead" id="flash"><%= value %></div>
<% end %>

make a new file called assets/javascripts/flash.js.coffee

put this in (beware of spaces, make sure all indentations are tabs):

jQuery ->

    $('#flash').hide().delay(800).fadeIn(800).delay(4000).fadeOut(800)