In one of the projects for children the client wanted to implement mini-piano for children to have the possibility to play simple melodies. And this piano was implemented.
For each sound separate flow with a media-player is launched which reproduces the necessary sound. Such a system should allow to reproduce several sounds at once - simultaneously.
Separate Key class was created to describe piano keys. This class kept data of sonund, which should be reproduced when pressing the key, and also kept the status of the key, if it pressed or not, plus some additional information.
data class Key(val rect: RectF, val sound: Int, val icons: List<Int>) {
var down = false
}
When creating View-component of piano a set of several black and white keys was created.
Amount of them was always small and always the same. This allowed to operate them as two lists of keys.
Standard method of callback onTouchEvent was used to process “pressing” and “releasing” of buttons.
This method was described in the parent class “View” from Android SDK. And if redefine method, it is possible to process such events as touching, release and move, that is when the finger of user touches the screen and then without taking off the surface is moved to any side.
override fun onTouchEvent(event: MotionEvent): Boolean {
val action = event.action
val isDownAction = action == MotionEvent.ACTION_DOWN || action == MotionEvent.ACTION_MOVE
for (touchIndex in 0 until event.pointerCount) {
val x = event.getX(touchIndex)
val y = event.getY(touchIndex)
val key = keyForCoords(x, y)
if (key != null) key.down = isDownAction
}
val tmp = whiteKeys + blackKeys
tmp.forEach { key ->
if (key.down) {
if (!soundPlayer.isNotePlaying(key.sound)) {
soundPlayer.playNote(key.sound)
invalidate()
} else releaseKey(key)
} else {
soundPlayer.stopNote(key.sound)
releaseKey(key)
}
}
return true
}
This method as a parameter receives object of MotionEvent type, which allows to define type of event, coordinates of formation, amount of touches and so on.
If looking at the redefined method then we will see, that if event type is ACTION_DOWN or ACTION_MOVE, then this is the event of touching the key, and this key should be displayed as pressed and try to play the sound.
After that coordinates of the event are defined for each index of touching, key is found according to these coordinates and tagged whether it is pressed or not. Then a temporary list is created with all keys included.
For each element of this list we check flag of pressing, and if it is set, then check of playing possibility is done. And if such possibility exists, sound is played and redraw of component for reflecting of pressed keys is initiated.
Elsewise “release” key method "releaseKey(key)" is opened.
private fun releaseKey(k: Key) {
postDelayed(PLAY_NOT_DELAY) {
k.down = false
invalidate()
}
}
"releaseKey" method plans pending setup of flag for key pressing in false redraw the component for reflecting state of pressed buttons.
What is wrong with the old piano?
- Pressing of 2 keys at the same time is not processed, that is, you can not press one key and then press another without releasing the first one.
- If you press the key and hold it for a long time, you can see key-flashing, the key gets white as released and then became dark as pressed.
- After releasing the long-pressed key its sound is played again.
- Fixing the piano - display the pressed keys correctly.
First of all I added logging to see which events happened. And it occurred that besides ACTION_DOWN and ACTION_MOVE there is also ACTION_POINTER_DOWN event.
If you read manual or find good site for newcomers (link will be in the end), then you may know that ACTION_DOWN takes place at FIRST touching. That is, when first touching takes place, the quantity of touchings becomes equal to 1.
For the rest touchings when there is at least one existing touching, ACTION_POINTER_DOWN event is activated. As a conclusion - not all events are processed.
There was no wish to add one more event into the expression of definition whether the key is pressed or not.
After reading a bit we clear up that except the index of touching there is also an identifier of touching, that is assigned to the touching and keeps unchanged until getting off the finger from the surface of the screen.
You can get this identifier by the index of touching and you can get this index for the events:
MotionEvent.ACTION_DOWN,
MotionEvent.ACTION_POINTER_DOWN
MotionEvent.ACTION_UP,
MotionEvent.ACTION_POINTER_UP
But it is not correct for the event ACTION_MOVE, for it we have to sort out all existing indexes.
Most of all, I didn’t want to break rendering following the tag of pressing. It might be a mistake all the same
The same task remains to define correctly what happens in the result of the event and calculate he flag correctly.
To get not confused I decided to divide processing of the events on three groups: pressing, release and move. And method took the following form:
override fun onTouchEvent(event: MotionEvent): Boolean {
when(event.actionMasked) {
MotionEvent.ACTION_DOWN,
MotionEvent.ACTION_POINTER_DOWN -> handleDownAction(event)
MotionEvent.ACTION_UP,
MotionEvent.ACTION_POINTER_UP -> handleUpAction(event)
MotionEvent.ACTION_MOVE -> handleMoveAction(event)
}
invalidate()
return true
}
So, we process separately each group of events and then launch redrawing of keys.
The first thing that came to my mind is to create a map where the keys will be identifiers of touchings, and meanings will be describing keys.
That is, when getting ACTION_DOWN and ACTION_POINTER_DOWN events we define the key where the event and identifier of touching occured. After that we save the key in the map according to ID key.
We check the flag of pressing before adding and if it is FALSE, set it TRUE and launch playing of sound. Why doing this way? Because people, especially children, are unpredictable and could press one key with two or three fingers, but we should process only one touch.
private fun handleDownAction(event: MotionEvent) {
val actionIndex = event.actionIndex
val pointerID = event.getPointerId(actionIndex)
val x = event.getX(actionIndex)
val y = event.getY(actionIndex)
val key = keyForCoords(x, y)
if (key != null)
pressKeyByPointer(pointerID, key)
}
private fun pressKeyByPointer(pointerID: Int, key: Key) {
touchIdToKey.put(pointerID, key)
if (!key.down) {
key.down = true
soundPlayer.playNote(key.sound)
}
}
We process the events of key release the similar way
private fun handleUpAction(event: MotionEvent) {
val actionIndex = event.actionIndex
val pointerID = event.getPointerId(actionIndex)
touchIdToKey[pointerID]
?.also { releaseKeyByPointer(pointerID, it) }
}
private fun releaseKeyByPointer(pointerID: Int, key: Key) {
touchIdToKey.remove(pointerID)
key.down = touchIdToKey.containsValue(key)
}
We do not lose here anything but should not forget that single key might be pressed with several fingers, so after deleting the meaning by touching ID, we check whether the key is pressed with a finger with a different ID.
And processing of ACTION_MOVE event consists of processing key release and pressing.
private fun handleMoveAction(event: MotionEvent) {
for (touchIndex in 0 until event.pointerCount) {
val x = event.getX(touchIndex)
val y = event.getY(touchIndex)
val key = keyForCoords(x, y)
val pointerID = event.getPointerId(touchIndex)
val pKey = touchIdToKey[pointerID]
if (pKey != null && key != null && pKey.sound != key.sound)
releaseKeyByPointer(pointerID, pKey)
if (key != null)
pressKeyByPointer(pointerID, key)
}
}
In this method we define the button where the event occurred for each index of touch. Then define the identifier of touching and check what finger with this ID had pressed earlier.
We read the value from our map for this. And if it is defined and sound that should be played does not correspond to the sound of the key, where the current event occurred, then we call the method of previous key “release” and then initiate pressing of the current one.
After changing onTouchEvent(event: MotionEvent) method reflection of touching and releasing the keys began to work correctly, but all sounds began to play only one time. Will fix this next time.
Good luck!
