You can almost always refuse to answer police questions, but depending on the circumstances it may produce somewhat different legal results.
Here is a general breakdown of your legal options when questioned by the police in three common scenarios:
1. If You're Stopped On the Street
The right to remain silent -- your most popular Fifth Amendment-based Miranda right against self-incrimination -- isn't required to be read or spoken to you until you are in police custody and are being questioned.
In Terry v. Ohio, the U.S. Supreme Court decided that police could make brief investigatory stops without probable cause and without violating the Fourth Amendment, assuming they have specific and articulable facts to justify the stop.
When stopped by the police on the street, you can always refuse to answer any of their questions, and you may ask if you are being detained. If officers respond that you are not being detained, you may stop conversation altogether and walk away -- though running away in a "high risk neighborhood" can potentially give officers reasonable suspicion to detain you.
2. If You're Asked to Come In for Questioning
You need to be in police "custody" in order for an officer to give you a Miranda warning, and consenting to come in for questioning technically does not count as custody.
So if you volunteer to walk into a police station without being under arrest, and choose to answer police questions, you do not need to be read your Miranda rights, the Supreme Court has held.
You can, however, ignore or refuse a request to come in for questioning -- but police may then choose to arrest you.
3. If You're Arrested
Once you have been arrested, you are certainly in police custody. As you know from your Miranda rights, you can invoke your Fifth Amendment right to remain silent.
Although you can't exactly walk away from police if you're under arrest, it is often in your best interest to not speak with them at all, as even idle conversation can lead to legally permissive evidence against yourself.