In 1968 there was a famous Computer Science article Go To Statement Considered Harmful by Edsger W. Dijkstra (of algorithm fame). It says in part:

My second remark is that our intellectual powers are rather geared to master static relations and that our powers to visualize processes evolving in time are relatively poorly developed. For that reason we should do (as wise programmers aware of our limitations) our utmost to shorten the conceptual gap between the static program and the dynamic process, to make the correspondence between the program (spread out in text space) and the process (spread out in time) as trivial as possible.

In early 21st Century America, pedestrian crosswalks may be marked or unmarked. Whether a crosswalk is marked is functionally based on the whim of the traffic department. A fuller discussion of issues about “how” to use crosswalks (from the Town of Brookline, Massachusetts) is here, but not “when” to use them, hence my use of the term “whim”, which says engineering studies are required, but does not have hard and fast rules about application.
Interesting the Brookline document asserts:

Marked crosswalks are viewed widely as “safety devices,” and most municipalities give the pedestrian the right-of-way when within them. However, there is strong evidence that these facts prompt many pedestrians to feel overly secure when using a marked crosswalk. As a result, pedestrians will often place themselves in a hazardous position by believing that motorists can and will stop in all cases, even when it may be impossible to do so. It is not unusual for this type of aggressive pedestrian behavior to contribute to a higher incidence of pedestrian accidents and cause a greater number of rear-end collisions. In contrast, a pedestrian using an unmarked crosswalk generally feels less secure and less certain that the motorist will stop and thereby exercise more caution and waiting for safe gaps in the traffic stream before crossing. The end result is fewer accidents at unmarked crosswalks.

Implicitly the document blames pedestrians for asserting their rights, rather than drivers for violating them.
I posit that if you are a trained, but human driver, whose “intellectual powers are rather geared to master static relations” you will generally respect crosswalks. You will believe, just as all stop signs are marked, all legal crosswalks are marked. As “our powers to visualize processes evolving in time are relatively poorly developed” you will disrespect unmarked crosswalks, since if they were legitimate, you reason, they would be marked. You may not even notice them if they come from side streets for which you have no stop sign of traffic signal. They only appear relevant when there is a person surprising you in the road. Hence you will be aggressive to pedestrians trying to cross at unmarked crosswalks, as you will (wrongly) believe you have the right-of-way. Pedestrians will in turn be intimidated as suggested by the Brookline document above. Research about driver and pedestrian behavior can be found in this paper by Mitman et al. It notes:

Driver yielding behavior was a statistically significant variable at all six observation sites. For all road types, pedestrians in the marked crosswalk were more likely than pedestrians in the unmarked crosswalk to have drivers immediately yield the right-of-way to them.

and

Average gap acceptance was a statistically significant variable at five of the observation sites. At all five locations, pedestrians in the unmarked crosswalk were more likely than pedestrians in the marked crosswalk to wait for larger gaps in traffic before crossing. This finding was consistent across all road types.

The empirical findings are sound as far as they go. I disagree with the recommendations.
The problem is inconsistent ambiguity.
Solution A. Mark all crosswalks.
If we were completely consistent about where pedestrians might be found, (i.e. crosswalks) that would be acceptable, drivers and pedestrians would both understand the law. It would be clearly spelled out to drivers where pedestrians might be, including smaller intersections that might otherwise be raced by. It would be bad from a pedestrian rights perspective, as it over channelizes walkers and gives too much power to cars.
By implication, it requires pedestrians to use only marked crosswalks. It in a sense delegitimizes jaywalking. It increases pedestrian travel times. As Peter Norton notes in Fighting Traffic:

“Before the American city could be physically reconstructed to accommodate automobiles, its streets had to be socially reconstructed as places where cars belong.” “Until then, streets were regarded as public spaces.” [Quoted in Planning Pool]

In practice, we will not mark all crosswalks. The vast majority of intersections in the US are unmarked, and no one wants to spend the money to mark them all. Hence if we claim to adopt solution A, we will in fact resign ourselves to inconsistent ambiguity (false certainty) or crosswalk markings.
Solution B. Unmark all crosswalks.
In contrast, if we were completely (i.e. consistently) ambiguous about where pedestrians would be, that would be good from both a safety perspective, and in the long run, a pedestrian rights perspective. While in the mixed environment, pedestrian might wait more, in the no crosswalks environment, pedestrians will be cautious where they are now reckless. But pedestrians would also be more assertive in more places (those without crosswalks now) as they would know that drivers would be also be more cautious. This strategy will make both drivers and pedestrians more aware of their surroundings since pedestrians might be anywhere. (See shared space.)
In addition to unmarking all crosswalks, we should put up periodic reminder signs/messages to drivers when entering new districts, leaving freeways, etc. that pedestrians have the right-of-way. We might put up markers where pedestrians have died to somber-up drivers. (Further, we ought to develop some hand-signal communication protocol so pedestrians can signal drivers they are about to enter the roadway. Reuben Collins has a nice discussion here.).
…
It is the false expectation of consistency that causes many of the 4,280 pedestrian deaths per year in the United States.
I strongly prefer Solution B. Do we have any examples of this in the United States over a widespread area? A single street with shared space would be insufficient to draw conclusions.
Comment: this is the same argument as about Class III Bikeways. Since Class III Bikeways give bicyclists no advantage, they imply to drivers that on any unmarked road, they have rights over bikes (when they don’t).
Comment: Yes I did see a driver yell at a pedestrian for crossing an unmarked crosswalk again today, and the intimidated pedestrian ran after trying to yield the road.