You can store references in another structure, but you probably don't want to do this most of the time, since it's a major headache to get ownership and lifetimes right. You shouldn't think of references as pointers, but you should think of them as "borrows": you are temporarily borrowing something from the owner, but you can only do so if the owner exists. So you need to statically guarantee that for as long as you borrow it, the owner must be alive, which gets tricky when you store that borrow somewhere in some data structure. In practice, references or borrows will be short-lived, and most operations on data will be done by the owner of that data.

Underneath, references are represented by pointers, but you shouldn't think of them as pointers, but rather as something you have borrowed, so in that sense it's different from C.

Also, Python does use references everywhere, it's just implicit, and depends on the type. Try storing a list in a class: you've just stored a reference to another structure. Most things (e.g. lists, objects) are passed and stored by reference, some types like integers are copied. In Rust, you can choose whether you want to pass by reference, copy or move ownership. At this point we're still at a high level of abstraction, we don't think so much about whether this will be pointers at the low level.

But my main point is that whether you use pointers, references, or whether it's implicit or explicit doesn't make a language slow or fast, it just defines how programs are written. Rust is very fast because it's compiled to machine code and doesn't do garbage collection or have any other overhead from a "runtime". Python is relatively slow because it's interpreted. You could argue that more manual control over references/pointers can make a language faster, but it's not the main contributing factor.