How do you know that Jimmy has truly changed?

O. Henry provides many proofs that Jimmy Valentine has changed. The entire story is about how Jimmy changes from a criminal into a solid citizen. The first direct proof of his change is stated in plain words by the author.

A young lady crossed the street, passed him at the corner and entered a door over which was the sign “The Elmore Bank.” Jimmy Valentine looked into her eyes, forgot what he was, and became another man. 

Is this possible? Well, it happened pretty fast, but many young men change when they fall in love. They grow up. They realize that there is a serious purpose in life, and for most men this involves getting married, having a family, becoming a provider, having a legitimate and secure profession or a trade, assuming a respectable place in society. All of this happens to Jimmy, as O. Henry illustrates, because of his love for the beautiful and high-principled Annabel Adams.

Early in the story O. Henry describes Jimmy's suitcase in detail. It is full of his specially designed safecracking tools. The suitcase is mentioned regularly right up to the end. It is really a symbol. It identifies him as an expert safecracker, but it is also a heavy burden. When he goes straight he wants to get rid of that burden. He offers to give it to a friend and tells him the following in a letter.

Say, Billy, I've quit the old business—a year ago. I've got a nice store. I'm making an honest living, and I'm going to marry the finest girl on earth two weeks from now. It's the only life, Billy—the straight one. I wouldn't touch a dollar of another man's money now for a million. 

Then at the climax Jimmy is still burdened with his heavy suitcase and his old identity. He makes a noble sacrifice by opening the suitcase inside the bank and rescuing a little girl who has accidentally gotten locked inside the supposedly burglar-proof vault. In doing this, he knows he is losing his new identity, the girl he planned to marry, his reputation as a solid citizen, and surely going to prison for a long stretch. This is the most solid evidence that he has really become a "another man." Fortunately, Ben Price the bank detective has seen Jimmy noble deed and is himself convinced that Jimmy has changed and is no longer a threat to society. Instead of arresting Jimmy and getting him sentenced to what could be twelve years in prison for three bank jobs, Ben lets Jimmy keep his new identity. When Jimmy resignedly and despondently offers to give himself up to the law, Ben Price responds:

“Guess you're mistaken, Mr. Spencer,” he said. “Don't believe I recognize you. Your buggy's waiting for you, ain't it?”

Jimmy really has become a different man in every way, including his name. He is no longer Jimmy Valentine but Ralph Spencer, respectable small-town businessman and pillar of the community.