= Upmpdcli == Introduction Upmpdcli is primarily an UPnP Renderer front-end to the MPD music player. It has a number of additional or alternative functions. The xref:UPNP[next section] explains the network audio terms (UPnP, Media Renderer, etc.) if your knowledge needs a little brushing up. Else, you can jump to the xref:UPMPDCLI-OVERVIEW[upmpdcli overview] section. Do not scrap your old amp yet, with 100 Euros (a Raspberry Pi and a high quality DAC I2S daugtherboard) you can transform it into a state of the art network streamer (my main one is 25 years old). === Security WARNING: The UPnP protocols have no security provisions. Upmpdcli is not audited for security issues, and, as far as I know, it is full of exploitable bugs. Do not run it on an Internet-facing host. [[UPNP]] == UPnP and all that === UPnP AV UPnP is a family of network protocols, designed for easily connecting _stuff_ in home networks (mostly). Its name ("Universal Plug and Play") was chosen because little or no configuration is needed for the different elements to discover each other and play together. The UPnP audio/video section (UPnP AV) implements a way for data stored on a network host (PC, NAS, etc.) to be displayed on another one (TV, audio streamer, etc.). Generally, there are four building blocks in a network audio solution: - An audio File/Data server stores the (bulky) audio data and makes it available to clients using an appropriate protocol (NFS, SMB, HTTP, etc. depending on client). - The directory/tags manager extracts audio tags from the files (usually accessing them directly or through a network file sharing protocol), and builds a database used for searching or browsing the collection. - Audio players (renderers) take encoded audio data and actually make sound, using some kind of DAC (Digital Analog Conversion) device, such as an internal or USB sound card. - The user interface is a remote control which sits in your hand (or it's the PC in front of you). You use the application which runs on it to choose songs from the database and control the playback. It interacts both with the tags database and player parts (usually not with the file server, except maybe to fetch cover art). .Network Audio image::pics/netaudio.png["Network Audio Elements"] In the UPnP world: - The tags manager is named Media Server (more precisely, the Content Directory service inside the Media Server device). Data is usually stored on the Media Server, but it could also live elsewhere. Data access is almost always performed through HTTP. - The player is named Media Renderer. It usually does not implement much user interface. - The remote control is named Control Point The good people who specified UPnP AV got something wrong: the playlist is owned by the Control Point, not by the Renderer. This means that the Renderer will not play the next track if the Control Point goes to sleep, and that havoc is almost guaranteed if several Control Points try to share a Renderer. This is the main reason for using OpenHome (see <>). [[OPENHOME]] === OpenHome The link:http://wiki.openhome.org/wiki/OhMediaDevelopers[OpenHome] family of open protocols were designed by link:http://www.linn.co.uk/[Linn] to correct the weaknesses in UPnP AV. They implement extended services over the base UPnP mechanisms. Principally, the OpenHome playlist lives on the Renderer. It will continue playing if the Control Points goes to sleep, and it can be shared. This is a compelling advantage and most well-known Control Points (e.g. BubbleUPnP, Kazoo, PlugPlayer...) support OpenHome. Beyond this, OpenHome also defines additional services, for example for accessing Internet radios. The different possible origins of audio data inside an OpenHome Renderer are named _Sources_, e.g. Radio Source, Playlist Source. Sources are switchable from any Control Point supporting OpenHome. OpenHome Sources are designated by a type and name, and a Renderer can have any number of them (for example, Upmpdcli uses additional sources to control its Sender mode). === Songcast link:http://oss.linn.co.uk/trac/wiki/Songcast[Songcast] is an open protocol designed by Linn for real-time network audio. Its only relationship with UPnP is that UPnP services are used to control the connections. Songcast allows playing the same audio on multiple players, with almost perfect synchronization, or capturing audio on a computer to play it on another system. The protocols links two types of entities: - A Songcast Sender generates an audio stream. - One or several Songcast Receivers receive and play the stream. Linn supplies free audio capture drivers functionning as Songcast Senders, for MS-Windows and Mac OS X. This enables playing the audio of *any* application (Windows Media Player, Spotify, Tidal, etc.) onto one or several Songcast Receivers, without any need for the applications to know about Songcast. === Gapless ? This is a relatively small issue, but people are very sensitive to it, it is important from some music pieces (Goldberg variations with gaps: not nice) and you will hear about it all the time if you read forums, so, a quick explanation follows. Some music tracks need to flow into each other when played consecutively, for example tracks on some live albums (the background noise from the audience never stops), or some classical music pieces. When playing from separate audio files, it is not trivial for a renderer to ensure a seamless flow: the audio hardware has an input pipe which must be kept full at all times, else it will stop, and a period of silence will ensue: not gapless ! To avoid this, the renderer has to know in advance which file it will process after the current one, to be able to begin filling the audio pipe with data from the next track before it empties at the end of the current one. There are two ways that this can be achieved: either the renderer is also in charge of the play queue, so it knows the whole sequence all the time (e.g. MPD, OpenHome, …), or the controller indicates the next track to the renderer before the current track finishes (e.g. UPnP SetNextTransportURI). [[UPMPDCLI-OVERVIEW]] == Upmpdcli overview Upmpdcli is primarily a UPnP Media Renderer frontend, using a slave MPD to actually process the audio data. It supports UPnP gapless track transitions and the OpenHome ohMedia services (including a Radio service to listen to Internet streams). .The Grand Upmpdcli Scheme of Things image::pics/upmpdcli.png["The Grand Scheme of Things"] Additionally: - Upmpdcli implements control functions for the Linn Songcast protocol, allowing the local machine to behave as a Songcast Receiver or Sender (with help from the closely related *sc2mpd* package, for processing the audio streams). - Upmpdcli provides a Media Server interface, currently used for accessing external streaming services (Qobuz, Tidal, Google Music). A local Media Server is in development, the motivation being that I see none which is both open-source and really satisfying (MinimServer is great but not open-source). Upmpdcli can be installed on most Linux systems. It is frequently used on small single board computers like the Raspberry Pi. As far as I know, all audio-oriented distributions currently use it as their UPnP Media Renderer interface, and it is also present in a number of commercial devices. Upmpdcli is mostly configured through values set in a configuration file but, for basic operation without a configuration file, a few of the main parameters can be set on the xref:COMMAND-ENVIRON[command line or in the environment]. Most parameters can only be set in the configuration file, which is designated with the +-c+ command line option. The usual location is +/etc/upmpdcli.conf+ The configuration file has a simple `name = value` format, and is fully described in the <>. upmpdcli has many configuration parameters, which may make the configuration intimidating. However, most parameters only relate to one function of the program and they are grouped in sections, many of which are probably not relevant to you. In each section, the most important parameters are presented first, and you probably will never need to bother with the bottom of the list. The <>, which are not typos ///////////////////////////////////// This section goes in a little more detail about miscellaneous aspects of the Media Renderer, and describe a few of the most used configuration parameters. See the <> for more details (I would very much like to link each parameter to its definition but Asciidoc won't let me do it, so they link to the top of the appropriate section.). If several instances of *upmpdcli* run on the same network, you will want to give them distinct names. The name which is displayed by most Control Points can be set with the <> configuration parameter. Some Linn Control Points (e.g. Kazoo) use another value, set with <>. Upmpdcli usually controls the MPD instance running on the same host. However, there may be reasons to do otherwise, and the host name and port where mpd should be reached can be set with <>, and <>. The Upmpdcli Media Renderer has two active interfaces by default: UPnP AV and OpenHome. Only OpenHome Control Points can share the renderer. If you use an UPnP AV Control Point, it must be the only one. This is not enforced, and misuse will result in miscellaneous weirdnesses. In some special situations, it may be useful to limit the interface to UPnP AV or OpenHome only (or disable both), which can be done with the <> and <> parameters. [[UPMPDCLI-RENDERER-FORMATS]] === Audio formats Upmpdcli can accept most audio formats supported by MPD, meaning about anything, including DSD. Upmpdcli normally checks that the format of a resource to be played is compatible with what it thinks MPD can do. This check is sometimes pessimistic and can be disabled by setting <> to 0. You should know that MPD has difficulties with some formats _when accessed through HTTP_ which is how the Media Server transfers the data. Specifically, WAV and AIFF files, especially with samples wider than 16 bits are a frequent source of trouble (because they are little used and little tested for streaming). Support will vary depending on the MPD versions and exactly what input plugins are configured (among *ffmpeg*, *libaudiofile* and *libsndfile*). Often, the same files play just fine locally, it's the combination of HTTP access and file format which causes problems. Raw PCM streams are another special case. The reason is that these streams do not, by definition, carry metadata to define the exact format (sample rate, bits per sample, number of channels, byte order). *upmpdcli* has no way to transfer these parameters to *MPD* (this is a limitation of the client protocol). The parameters can be transferred from the Media Server to the player along with the MIME type though. In consequence, only recent versions of Upmpdcli and MPD (0.20 and later) do support audio/L16, but not with any Media Server (it must output the audio formats parameters with the MIME type). See this issue for more details: https://www.lesbonscomptes.com/upmpdcli/github-issues/upmpdcli-html/issue-36.html In general, there are few reasons to use these linear formats, when FLAC will produce exactly the same bits, with less network load (which largely compensates the small additional CPU load). There are situations where the linear formats are needed though, and where it may be necessary to use the right MPD and plugins versions. [[radio-definitions]] === Radio station definitions Upmpdcli versions after 0.13 implement an OpenHome Radio service which allows selecting and listening to internet radio stations. This facility uses Python 2.x, which must be available on the system for the radio links to work. Radio stations can be defined in the configuration (at the end because of the use of section indicators), or in in a separate file by setting the <> parameter in the main configuration. Radio channels can be accessed by selecting the _Radio_ Source from an OpenHome Control Point. Example entry: ---- [radio Radio Teddy] url = http://opml.radiotime.com/Tune.ashx?id=s80044 artUrl = http://cdn-radiotime-logos.tunein.com/s80044q.png artScript = /path/to/some/script metaScript = /path/to/some/other/script ---- The section name must begin with +radio+, the rest will be displayed as the station name. The section has several possible entries. Only +url+ is mandatory, it points to the playlist or stream. If set, +artUrl+ is a link to a static icon for the radio (displayed as album art). If set, +artScript+ is the path to an executable script which can retrieve the image URL for the currently playing title. Some radios (e.g. link:https://www.radioparadise.com/rp_2.php?#[Radio Paradise]) publish the album art for the currently playing title. The details vary. The +artScript+ parameter, if set, should point to an executable script which prints this dynamic art Uri to stdout. The image will supercede the radio logo, to be displayed by control points. Beware that the path to the script must be accessible by the _upmpdcli_ user, which may not be the case for your home. +/usr/local/bin+ is your friend. As a very rough example here follows a command which would retrieve the Radio Paradise URI (as long as they don't change their format, a proper approach would use an XML parser of course): curl --max-time 5 -s http://radioparadise.com/xml/now.xml | \ grep '' | sed -e 's///' -e 's!!!' Also note that upmpdcli does not set a timeout for the +artScript+ execution. A reasonable value should be used inside the script, to avoid freezing upmpdcli forever. If set, +metaScript+ is the path to an executable script which can retrieve the metadata (possibly including art Url) for the currently playing title. The script should output the metadata in JSON format. Example output (the newlines and blank space are just here for readability): ---- { "title":"The title of the current track", "artist":"The artist playing", "artUrl":"https://www.somesite.com/path/to/image.jpg", "reload":103 } ---- The +reload+ value gives the number of seconds after which the script should be re-executed (the clever radio sets this to the remaining song time). By default, the script is executed every 10 S. Complete exemple with metadata script: ---- [radio FIP (Paris)] url = http://direct.fipradio.fr/live/fip-midfi.mp3 artUrl = http://www.fipradio.fr/sites/default/files/fip-quadri-filet.png metaScript = /usr/local/bin/fip-paris-meta.rb ---- And the script (this one in Ruby, thanks to link:https://github.com/distler[Jacques Distler]): ---- #!/usr/bin/ruby require 'net/http' require 'json' uri = URI('http://www.fipradio.fr/livemeta/7') response = Net::HTTP.get(uri) songs = JSON.parse(response)['steps'].values now = Time.now songs.each do |song| song_end = Time.at(song['end']) if (song['embedType'] == 'song' && song_end >= now && Time.at(song['start']) <= now) title = song['title'] ? song['title'] : '' artist = song['performers'] ? song['performers'] : song['authors'] metadata = {'title' => title.split.collect{|s| s.capitalize}.join(' '), 'artist' => artist.split.collect{|s| s.capitalize}.join(' '), 'artUrl' => song['visual'], 'reload' => (song_end - now + 1).to_i} puts JSON.generate(metadata) break end end ---- [[UPMPDCLI-SONGCAST]] == Upmpdcli and Songcast Upmpdcli implements the Receiver UPnP service, and uses an auxiliary process (*sc2mpd*) for transporting the audio data. *sc2mpd* is based on the sample program which comes with the Linn Songcast OpenHome open source implementation Upmpdcli can also manage a Sender subsystem, which is implemented by using a separate *mpd* instance sending audio to an *mpd2sc* command (part of the *sc2mpd* package). The latter is a modified version of the OpenHome WavSender sample program. This allows playing the usual Upmpdcli playlist or a a radio channel on several synchronized players, but also doing the same thing with a captured analog source (e.g. *arecord* output). NOTE: You should know that it is possible to control the Songcast Sender from another local network PC to snoop on what you are listening (Radio or Playlist). This is detectable from the Renderer state, but not obvious. In any case, the playlist itself is public (there are no privacy provisions in UPnP), so this is probably not a major additional issue. The system will not capture anything besides what *mpd*, or an explicitely setup additional source are playing (e.g. Skype phone conversations are out of reach). [[UPMPDCLI-SONGCAST-COMPAT]] === Upmpdcli Songcast Control Point compatibility Upmpdcli Senders and Receivers can be mixed with Linn ones and controlled from Upplay or Bubble DS (an maybe other control points). Neither Linn nor upmpdcli receivers can be linked to an upmpdcli Sender using Linn Kazoo (tested with release 4.11.226), the reason is unknown. Upmpdcli Receivers before release 1.2.14 are not visible in Linn Kazoo. Later versions can be linked, and stream, from a Linn Sender. Linn Kazoo can control the playlist in an upmpdcli instance which has been put in Sender mode by another CP (i.e. currently using the PL-To-Sender source). The Lumin IOS app will reset the Source to Playlist when connecting, so it is unusable in this context. [[UPMPDCLI-RECEIVER]] === Upmpdcli Songcast Receiver *sc2mpd* can play the *Songcast* audio stream in two modes: - By directly using the *alsa* audio driver. - By using *mpd* to play the stream offered from a local HTTP interface (hence the *sc2mpd* name) .Upmpdcli Receiver in `alsa` and `mpd` modes [options="header"] |============= |`alsa`|`mpd` |image:pics/upmpdcli-receiver-alsa.png[Upmpdcli Receiver, alsa mode]|image:pics/upmpdcli-receiver-mpd.png[Upmpdcli Receiver, mpd mode] |============= What approach is used is decided by the <> configuration parameter, which can be set to _mpd_ or _alsa_. The _mpd_ mode is the default, because it needs no configuration, but there are *multiple* problems with it. If you go the _alsa_ route, you will need to set <> in the configuration, but you probably had to set it in the MPD configuration too, so this may not be too much of an issue. NOTE: when using the _alsa_ approach, *sc2mpd* will open the audio driver directly. The *sc2mpd* process is started by *upmpdcli*, which normally runs with user id _upmpdcli_. A frequent cause preventing the Receiver from working is that the driver open fails because _upmpdcli_ user has no permission to access it. One solution is to create an _audio_ group (if it does not exist), make the devices writable by _audio_, and add _upmpdcli_ (and others) to the _audio_ group. When using *mpd*, more bufferisation occurs and there may be a significant delay (up to around 10 S) between the time when Songcast is activated and the time sound appears. NOTE: When using _mpd_, from a Mac (24 bits audio) you need an appropriately configured and recent MPD version (usually configured with `--disable-audiofile`), else you risk producing high volume noise. As Debian and Ubuntu tend to lag quite bit on MPD progresses, I have set up link:downloads.html#mpd[backport repositories] for appropriately configured recent mpd versions, for Ubuntu, Debian i386/amd64 and Raspbian. Unfortunately, not only the MPD version is significant, there are sometimes also problems in the input plugins libraries. Given the bufferisation and delay control issues when going through MPD, only the _alsa_ method is usable in multi-room configurations. Even with a single player, the _mpd_ method will experience skips from time to time. The reasons are explained in the xref:UPMPDCLI-MULTIROOM-SYNC[multiroom synchronization section]. The following steps are taken for establishing a Songcast connection: - If *upmpdcli* finds an executable *sc2mpd* command in the PATH when starting up, it advertises a Receiver service. - The *Songcast* application on the desktop finds out about the Receiver through the normal UPnP mechanisms and can be instructed to use it. It then tells the Receiver in *upmpdcli* to start playing. - *upmdpcli* starts the *sc2mpd* process, which gets ready to receive data through Songcast, and either play it directly or make it available through HTTP. - If the _`mpd`_ method is in use, *upmpdcli* instructs *mpd* to play the URL for the *sc2mpd* output. WARNING: There is *no software volume control* for the *upmpdcli* Songcast Receivers for now: use either a local mixer or the little round things on the pre-amps. Set the volume low when experimenting ! NOTE: Songcast can use IP multicast for lower load on the network when playing on several hosts. Unfortunately, multicast and WIFI don't mix well in many cases. If you have wireless Receivers experiencing sound drop issues, try selecting unicast in the Songcast advanced configuration panel on the desktop. [[UPMPDCLI-SENDER]] === Upmpdcli Songcast Sender Upmpdcli Sender mode allows you to broadcast the Playlist or Radio source (or the output of any process which can write to a Fifo, e.g. *arecord*) to other Songcast Receivers. The local *upmpdcli* plays through its Receiver too, in order to achieve good synchronisation. Unlike the Songcast applications on Windows or Mac, the audio is not captured from the driver, so that you won't be able to cast other applications transparently (but there are specific workarounds). You can set *umpdcli* in Sender mode from an OpenHome Control Point, by selecting the Source of type `Playlist`, name `PL-to-Songcast` or type `Radio`, name `RD-to-Songcast` (and others if using the extension mechanism). With link:http://www.lesbonscomptes.com/upplay/index.html[upplay], for example, you can select the source from the `File`->`Select Source` menu entry. The *sc2mpd* configuration *must* be set with +scplaymethod = alsa+ for the `XX-to-Songcast` sources to appear. See the xref:sclogfilename[configuration section], and the xref:UPMPDCLI-MULTIROOM-SYNC[section about synchronisation]. When entering Sender mode, *upmpdcli* will start an auxiliary MPD process (after stopping the main one), configured to send audio to the *mpd2sc* OpenHome Sender process. It will then also start its own Songcast Receiver mode, transfer the playlist, stop the main MPD and start the auxiliary one... The resulting state is that the *upmpdcli* instance can be managed in Playlist mode from the Control Point. The audio is played locally through the Songcast Receiver. Other Receivers can be connected and will play in good synchronization. image::pics/upmpdcli-sender-receiver.png["Sender/Receiver mode"] This looks complicated, but in practise, starting the mode and connecting other Receivers from the *upplay* control panel is quite straightforward. [[UPMPDCLI-SENDER-EXTSOURCES]] ==== Managing external sources In the Sender modes described above, *upmpdcli* (or rather its slave *mpd* process) is the source of the audio stream. The *mpd2sc* program (which is just a slight modification of a Linn sample) reads the *mpd* output from a FIFO and sends it to Songcast Receivers (including the local one). There is nothing special with the *mpd* output (it is just PCM audio), and it is also possible to set things up for *mpd2sc* to read from another source. This is made reasonably easy because the pipe is actually built by a script (the default one is named 'scmakempdsender'), and *upmpdcli* has built-in functionality for choosing different ones (creating OpenHome sources for each script it finds in a specific directory). There is a sample script in the distribution, which itself is flexible enough to allow connecting several inputs without programming (anything *arecord* can read). Still the details are a bit too much involved for the main body of the manual, so I have moved them to an xref:SENDER-EXTSOURCES[annex]. [[UPMPDCLI-MULTIROOM]] === Songcast multiroom configurations and synchronization issues Multiple Songcast Receivers can connect to the same Sender, for synchronized playing of a the same audio stream. The Mac and Windows Songcast applications only let you activate one Receiver. The easiest way to manage multiple connections is to use an appropriate control point, such as Upplay on Linux and Windows, BubbleDS on Android, or Linn Kazoo (multiple platforms). For the record, the *upmpdcli* package also includes the *scctl* command line utility, which can list the state of the local Songcast Receivers, make a Receiver play from the same URI as another one (for building multi-room groups), or return a Media Renderer from Receiver to normal operation. There is also a xref:SCWEB[small Web application] which can be accessed from a desktop web browser to control the groups. This has only be tested with *upmpdcli* as Receiver implementation, but I'd guess that there is a good chance it would work with others. Quite frankly, I think that you'd better use the Control Point approaches if at all possible. [[UPMPDCLI-MULTIROOM-SYNC]] ==== Synchronisation issues The short version is: for good syncrhronization, all *sc2mpd* instances must be configured to play directly to the audio driver with +scplaymethod = alsa+. See the xref:sclogfilename[configuration section]. Longer version: *Songcast* is a real-time audio stream. As the Sender and Receiver sample clocks (the 44.1 or 48 KHz clocks) are independant, audio reproduction on the two systems will slowly drift. If nothing is done, after a time, the Receiver will have to skip samples or add a period of silence (depending if its clock is slower or faster), which is quite audible and ennoying, and will happen "from time to time", depending of how much the clocks differ. The only way to control this is to adjust the rate of reproduction on the Receiver, which can be done in two ways: - *Linn* hardware uses timestamps embedded in the audio stream to adjust their hardware sample clock. - *sc2mpd* in _alsa_ mode uses sample rate conversion to adjust the stream. - *sc2mpd* in _mpd_ mode can't do a thing about it. This is not specific to *Songcast* of course, all real time audio network transports have to do something similar (e.g. Roon does individual stream adjustments on the server). Independantly of the clock issue, all Receivers should use approximately the same amount of buffering for the audio to be reasonably synchronous (with no more shifts than moving around would produce anyway). This is impossible to achieve when going through *mpd*, and the second reason why *sc2mpd* must be set in _alsa_ mode for multiroom setups. In _mpd_ mode, the Receivers can be out of sync by several seconds. [[UPMPDCLI-MS-STR]] == Support for Streaming Services === Streaming Services and UPnP Streaming services like Qobuz or Tidal give access to a huge catalog of music through a WEB interface (also usable by programs). The only thing that they have in common with UPnP is that the actual audio stream is obtained through HTTP. However, given their importance, they can be accessed through many UPnP Control Points and Renderers. There are three main ways to integrate the streaming services in an UPnP framework, and knowing which way you are using will avoid confusion: - The first approach is to have a UPnP Media Server act as a proxy for the service: the catalog is browsed through a standard UPnP interface (the Media Server translates this to the service API behind the scenes), and the audio URIs are also accessed in an almost standard way (after a bit of magic inside the Media Server). The one inconvenient of this approach is that the special functions of the service (like managing the user favourites lists) are mostly inaccessible. The advantage is that any naive UPnP Control Point can access the streaming service. The Media Server inside upmpdcli xref:UPMPDCLI-MS-STR-MS[implements this method]. - The second approach is to do everything inside the Control Point. The CP uses the service API for browsing (and possibly do things like managing favourites). The CP also mediates the HTTP access to the audio streams (the URIs the CP sends to the Renderer point to itself, and it arranges to forward the data from the service). This is implemented, for example, by Bubble UPnP. The drawback of this approach is that the CP can't go to sleep or out of reach, else the music will stop (maybe at the next track). - The third approach is to distribute the function between CP and Renderer. The Control Point performs the browsing and access to special functions, and the Renderer knows how to access the service streams. This is the OpenHome approach and can be considered the best of both worlds, at the price of complexity and compatibility issues. This method is implemented by the Kazoo and Lumin Control Points, and a number of Renderers, the Linn and Lumin ones of course, but also, xref:UPMPDCLI-MS-STR-CRED[lately by upmpdcli]. [[UPMPDCLI-MS-STR-MS]] === Accessing Streaming Services through the upmpdcli Media Server The Media Server function in Upmpdcli (1.2.0 and later) gives access to external streaming services: Qobuz, Tidal, and Google Music at the moment. You will need a valid account on any service that you want to use. When the function is activated, *upmpdcli* creates a separate auxiliary Media Server, with the sole purpose of presenting the streaming server catalog. You will see it appear on the network if streaming service parameters are present in the configuration file. The Media Server name is the +friendlyname+ for the Upmpdcli instance, with _-mediaserver_ appended. Streaming is not limited to the local Upmpdcli Renderer, and the Media Server will happily send music to other Renderers. Two conditions must be true for the Media Server to appear with an entry for a streaming service: - The code for the service plugin must be installed. This comes generally in a separate package (e.g _upmpdcli-qobuz_). - The user variable for the service must be present in the configuration file (e.g +qobuzuser+). Some other parameters (e.g. a password) may be needed for the access to work. NOTE: The streaming services passwords are stored in the upmpdcli configuration file ('/etc/upmpdcli.conf'). The installer does not make this file r/w by owner only. This will be fixed, but for now, you should probably do it after setting the password: +chmod 600 /etc/upmpdcli.conf+ Actually, it would quite probably be a good idea not to use the same account for your mail and music uses. NOTE: I have no reason to believe that upmpdcli is technically mis-using the streaming services in any way, but it is definitely not approved by any of them, and you are using it under your own responsibility. ==== Notes on the Tidal interface [NOTE] =================================== There seems to be problems when streaming Tidal lossless/FLAC streams, with the default MPD configuration. A preliminary and rather superficial enquiry seems to indicate that this is caused by too much buffering by MPD. Adding the following parameters to the MPD configuration file (/etc/mpd.conf) reduces the pre-fetch period from 20 S to 10 S and seems to fix the issues for me: audio_buffer_size "2048" buffer_before_play "20%" ==================================== The Tidal module code is is based on the link:https://github.com/tamland/kodi-tidal[Kodi Tidal add-on] and the link:https://github.com/tamland/python-tidal[Tidal API], by Thomas Amland (also using other useful modules by Thomas). I was able to reuse most of the Kodi code. The Tidal api module was recently updated and the login code comes from link:https://github.com/arnesongit/kodi-tidal[this Git repository]. The software is not officially supported by Tidal, and may just stop to work one day... It only works with a valid Tidal subscription. The configuration parameters are described in the <> ==== Notes on the Qobuz interface This is based on the service API from the link:https://github.com/tidalf/plugin.audio.qobuz[Kodi add-on], by Joachim Basmaison and Cyril Leclerc, with the upper-level code adapted from the Tidal add-on. The WEB API is not officially supported by Qobuz, and may just stop to work one day. NOTE: the Qobuz search does not allow searching a specific field (like the artist name or song title). All objects matching a query in any field are returned. This means that, e.g., the field choice of upplay search is ignored. You need a valid Qobuz subscription. The configuration parameters are described in the <> [[upmpdcli.googlemusicnotes]] ==== Notes on the Google Music interface This is based on the link:https://github.com/simon-weber/gmusicapi[gmusicapi] Python package by Simon Weber, with upper-level code from the Tidal Kodi add-on. As for the other services, the API used is not officially supported and may stop to work one day. This requires the installation of the external *gmusicapi* Python package. link:http://unofficial-google-music-api.readthedocs.io/en/latest/usage.html#usage[See the installation notes here]. NOTE: gmusicapi needs Python 2.7.9 or better (e.g. the standard Python version on Ubuntu Trusty is too old). Older Python versions *will* crash. You will need a valid subscription for most functions. I'm a little lost in the different google authentication/sign-in methods, but if you are using two way authentication, there is apparently link:https://www.lesbonscomptes.com/upmpdcli/github-issues/upmpdcli-html/issue-87.html[a way to make things work, using an app password]. The configuration parameters are described in the <> [[UPMPDCLI-MS-STR-CRED]] === Accessing the streaming services with OpenHome from Kazoo or Lumin This only works for Qobuz and Tidal. The corresponding media server module must be installed, but no other configuration should be needed on the renderer. The media service icons will appear in the CP interface, and the service will be accessible after entering the login and password. NOTE: the user name and password will be stored on the renderer in an insecure way, meaning that if someone steals the machine or obtains root access, there is nothing to prevent them from getting the credentials. Of course Kazoo or Lumin can also access the services through the media server interface (after configuring upmpdcli, see the xref:UPMPDCLI-MS-STR-MS[previous section]). [[UPRCL]] == Upmpdcli local Media Server: uprcl The current (2017-04) upmpdcli code base includes a Media Server function for local music files, analog to MinimServer or ReadyMedia/MiniDLNA. The plugin module is named uprcl. Why yet another Media Server ? MinimServer is great, and free, but not open-source. ReadyMedia is nice too, but audio file format support is incomplete (there was a patch for DSD but it was never merged apparently), I've always found it a bit clunky (for example: no incremental updates), and development does not seem very active. The new upmpdcli Media Server is at most beta quality. It probably has even less format support than MiniDLNA... But it's written in Python, and I have good hope that it can evolve faster. Some features: - Mostly written in Python, except for the pieces shared with the streaming plugins: UPnP interface proper, and plugin management, which are C++, but should not need a lot of changes. - Tries to mimick the MinimServer tree view, which I find quite nice. We're not quite there yet, many functions and niceties, and the customization capability are missing. - Uses Recoll and python-recoll (hence the uprcl name) for harvesting the metadata and searching. Recoll already had almost everything needed in this area, no development needed. It can perform incremental updates and file system monitoring. - Uses Python sqlite3 (on a memory DB) for the tags tree. Installation and configuration: have a look at the xref:uprcluser[configuration variables] NOTE: 'python-mutagen' which is optional for default Recoll installations is required for uprcl (this is for the case where you install by hand, the package takes care of it). [[UPRCL-SEARCH]] === Upmpdcli/Uprcl searches The UPnP Media Server search function is based on a specific query language. This language is quite verbose, and most control points will not let you access it directly. Instead, they will usually provide a generic entry field in which you enter search terms. Behind the scenes, they generate an UPnP search language string. This could be for example something like: `upnp:class = "object.item.audioItem" and dc:title contains "heaven"` If it supports search at all, the Media Server will then interpret the query string and return the results. If the Media Server is Upmpdcli/Uprcl, there is another step: the UPnP query string is translated into a Recoll one and Recoll is called to perform the actual search. There is no way to directly enter a Recoll search through the control point. However, two important Recoll capabilities should still be visible through the layers: .Stemming Stemming is the capability to expand a search to the common derivatives of a word. For example, a search for `floor` would be expanded to also look for `floors`, `flooring`, `floored`, etc. By default, stemming is enabled for Uprcl search. It can be turned off for any term by capitalizing the term (Entering `Stones` will only search for `stones`, not `stone` or `stoned`). Uprcl/Recoll searches are case-insensitive apart from this usage of capitalization. .Phrase searches By default, Uprcl/Recoll searches do not observe the order of terms: `red house` will be matched by a search for `house red` (or `houses red`). You can change this behaviour by quoting a sequence of words. A search for `"red house"` will only look for a text containing the words in order. This supposes that the Control Point does not change the user entry, but this is the case for at least some of them. Stemming expansion does not occur within phrase searches. Other Recoll capabilities, such as proximity searches should also be accessible, but probably less useful when searching music titles or artist names. [[UPMPDCLI-CONFIGURATION]] == Upmpdcli configuration file reference include::upmpdcli-config.txt[] [[UPMPDCLI-PACKAGES]] == Installation from packages The link:https://www.lesbonscomptes.com/upmpdcli/downloads.html[upmpdcli downloads page] has a number of pointers to binary packages for a number of common distributions (Debian, Raspbian, Ubuntu, Fedora) and architectures (ARM and x86). [[UPMPDCLI-BUILDING]] == Building from source === Building libupnpp and upmpdcli on Linux For building from source, you will need a C\++ compiler with full C++11 support, and the development packages for *libupnp* version 1.6, *libcurl*, *libmpdclient*, and *libexpat*. Also the Python modules for streaming service support use the *python-requests* package, so you may need to install it (it is only needed at run time). Also *gmusicapi* for Google Music. If you are using the source from the git repository, you will also need the *autoconf*, *automake*, *libtool* trio. Use the +autogen.sh+ script to set things up. The *libupnpp* library, which used to be part of *upmpdcli*, has been separated, and you need to build it first. So you need to either clone two git repositories: https://opensourceprojects.eu/p/libupnpp/code and https://opensourceprojects.eu/p/upmpdcli/code or download the release tar files from the the link:https://www.lesbonscomptes.com/upmpdcli/downloads.html[upmpdcli download page] Once the source is extracted, the procedure is standard and there are currently no specific configure options: # Only for git source sh autogen.sh ./configure --prefix=/usr --sysconfdir=/etc make sudo make install Which you should apply first to the *libupnpp* source, then to *upmpdcli*. If you omit the `--sysconfdir=/etc` part, `upmpdcli.conf` will end up in `/usr/etc/`, which is ok, but confusing, as package installers put it in `/etc/` ==== Boot time startup *upmpdcli* will try to change its `uid` to user `upmpdcli` if it is started by root. It will refuse to run if the user does not exist. If started by `root`, *upmpdcli* will also write its process id to `/var/run/upmpdcli.pid`. There are boot-time startup scripts in the `debian/` directory inside the source tree (for Debian/Ubuntu/Mint/Raspbian etc.). There is also a systemd service file under `systemd/` (for Fedora et al.). The boot scripts are installed by the pre-built packages, so will just have to edit the configuration file after installing them, all the rest should just work. === Building libupnpp and upmpdcli on Mac OS X I don't run Mac OS X and I can't support this directly. However a kind user has provided a link:https://opensourceprojects.eu/p/upmpdcli/tickets/3/[build recipe] based on homebrew. You will first need to apply the patches from link:https://www.lesbonscomptes.com/upmpdcli/files/2017-12-upmpdcli-macos-build-patches.tgz[this file]. Then: ---- brew install mpd brew install autoconf brew install automake brew install libtool brew install libupnp libmpdclient expat jsoncpp libmicrohttpd expat curl brew install pkg-config brew install expat export PKG_CONFIG_PATH=/usr/local/opt/expat/lib/pkgconfig configure --prefix=/usr make ---- === Building OpenHome and sc2mpd There are two parts in building *sc2mpd*: - Building the Openhome libraries - Building *sc2mpd* proper First download a link:http://www.lesbonscomptes.com/upmpdcli/downloads.html[*sc2mpd* release] or clone the *sc2mpd* https://opensourceprojects.eu/p/sc2mpd/[Git repository]. ==== Building the OpenHome libraries Building the Openhome libraries is a bit of a black art for a number of understandable reasons: - There are no released distributions for the OpenHome libraries, but only a number of git repositories for the different components. The state of the repositories is not completely consistent at all times (there will be moments when a component will not build with another). - The OpenHome build system is very complicated because it works on multiple platforms and multiple language bindings. The *sc2mpd* source comes with a shell script named _ohbuild.sh_ to help with this. It has several functions: - Cloning the OpenHome and checking out verified revisions - Applying a few minor patches to ensure that the build will work in a simplified environment (e.g. without a c# compiler). - Creating a tar file of a compact trimmed tree. - Running the build either from the cloned tree or the tar file. The tar functions are mostly useful to help with building packages. For a normal build, the procedure would be as follows: cd sc2mpd mkdir /my/place/for/openhome sh ohbuild.sh -b /my/place/for/openhome ==== Building sc2mpd *sc2mpd* depends on a number of libraries: - The link:http://www.gnu.org/software/libmicrohttpd/[microhttpd] library. Install the development and runtime packages which are currently named _libmicrohttpd-dev_ and _libmicrohttpd10_ on Debian-derived systems (use _libmicrohttpd_ and _libmicrohttpd-devel_ for Fedora). - The link:http://www.mega-nerd.com/SRC/[libsamplerate] library (_libsamplerate0_ on Debian, _libsamplerate-devel_ for Fedora). - The *libasound* Alsa interface library (_libasound2-dev_ on Debian, _alsa-lib-devel_ for Fedora). Once OpenHome is built and the auxiliary library development packages are installed, build *sc2mpd*, using the following commands inside the _sc2mpd_ directory: sh autogen.sh ./configure --prefix=/usr --with-openhome=/my/place/for/openhome make sudo make install The build uses static Openhome libraries, so you can move the executable to another machine without needing the Openhome directory (don't forget to install the _libmicrohttpd_, _libsamplerate_ and _libasound_ runtimes though). After restarting *upmpdcli*, it should advertise the _Receiver_ service and appear in the Songcast Sender menus. ==== Building a Debian package from an sc2mpd tar distribution Here follows a synopsis of building a debian package for an sc2mpd tar distribution. You'll need to replace the 1.1.1 and 20151223 versions with whatever you are building. Note that the 1.1.1 value must match what you have in the Debian changelog (or update the changelog). The $distdir and $sc2mpd_src values stand for places where you store the tar files and the sc2mpd (git) source tree. mkdir sc2mpd_build cd sc2mpd_build cp $distdir/sc2mpd-1.1.1.tar.gz sc2mpd_1.1.1.orig.tar.gz cp $distdir/openhome-sc2-20151223.tar.gz sc2mpd_1.1.1.orig-openhome.tar.gz tar xzf sc2mpd_1.1.1.orig.tar.gz cp -rp $sc2mpd_src/debian sc2mpd-1.1.1/ cd sc2mpd-1.1.1 mkdir openhome cd openhome tar xzf ../../sc2mpd_1.1.1.orig-openhome.tar.gz cd .. debuild [[SC2MPD-INSTALL]] == Annex: Songcast installation walkthrough If you want to play from a Mac or Windows machine, install the link:http://www.linn.co.uk/software#songcast[Songcast application] NOTE: Not all http://oss.linn.co.uk/Releases/Songcast/Davaar/[Songcast capture application versions] work well with upmpdcli. Lately (03-2017), the latest version (4.8.535) seems to work fine. In the past, I've had good luck with 4.4.190 but not with the 4.5 ones (it is possible to get them to work with *upmpdcli* from the *upplay* control panel, but the Windows app claims that the *upmpdcli* receiver is unavailable.). I could not get IP multicast to work with WIFI Receivers (the sound drops constantly). There are well-known problems with multicast and WIFI (see for example http://superuser.com/questions/730288/why-do-some-wifi-routers-block-multicast-packets-going-from-wired-to-wireless[this superuser.com question] for detailed explanations). This seems to be dependant on the WIFI hardware (router/access points) used, so maybe you will have more luck than me. If some of your Receivers use WIFI, and you experience sound issues, check that "Unicast" is selected in the Songcast configuration "advanced" panel on the desktop. Under most conditions, WIFI data rates should be more than sufficient to transport Songcast streams (a bit over 1 Mbit/S for 48k/24bits, 700 Kbits/S for 44.1k/16 bits). However, Songcast is probably best transported over a wired connection. If you are doing this over WIFI and experiencing glitches, the wireless is the first suspect. If the Sender is a *upmpdcli* host, you should probably also install the link:http://www.lesbonscomptes.com/upplay/[upplay] Control Point somewhere. Recent upplay versions include a tool to control Sender and Receiver instances. This is less necessary if the Sender is on Windows or Mac, as the Songcast application on these platforms includes a limited connection tool (no multiroom). Download and install the *upmpdcli* and *sc2mpd* packages for your Linux system. See link:http://www.lesbonscomptes.com/upmpdcli/downloads.html#packages[the download page here]. When using a single Receiver, things should just work, the _UpMpd_ Receiver should appear in the Songcast app on the host (or in the *upplay* control panel). WARNING: There is *no software volume control* for the *upmpdcli* Songcast Receivers for now: use either a local mixer or the little round things on the pre-amps. Set the volume low when experimenting ! NOTE: The receiver side does not work too well on Raspberry Pis using an USB DAC (as far as I could see). The reason may be that the data rate of the uncompressed stream is just too much for the poor Pi Ethernet/USB chip. Things work just fine with an I2S DAC (or on HDMI probably). When on WIFI, the results are sometimes irregular too, depending on the quality of the connection. If you have no Ethernet cabling, Ethernet over power lines may work better than WIFI. The best combination of network and DAC link is definitely Wired and I2S (or HDMI). When using multiple Receiver instances, you need to at least customize their names, and you want to take care of synchronization issues. You will need to take a look at a bit more documentation: - xref:friendlyname[+friendlyname+] - xref:UPMPDCLI-MULTIROOM-SYNC[audio synchronization issues and *sc2mpd* configuration]. [[SENDER-EXTSOURCES]] == Annex: Setting up external audio sources for the Songcast Sender The *upmpdcli* Sender modes are controlled by switching OpenHome Sources. By default, and if the *sc2mpd* package is installed, *upmpdcli* will create 'PL-to-Songcast' and 'RD-to-Songcast' sources for broadcasting its playlist or radio streams to Songcast. As explained in the main body, there is nothing magical with this: *mpd* just streams to a FIFO which is read by the *mpd2sc* command, which then broadcasts it. This pipe is set up by a script (_scmakempdsender_). Under certain conditions, if *upmpdcli* finds other scripts in a designated directory, it will create more sources, and start them when the corresponding source is selected (of course, it expects the script to create a Songcast Sender pipe). The *upmpdcli* source comes with a sample script which sets things up for *mpd2sc* to read *arecord* output, allowing multiroom playing of an external analog source (line-in). The sample script is flexible enough to manage different inputs without code changes, and there are comments at the top which give detailed explanations. The parts of more general interest will be a bit expanded here (see below). If you want to use the script itself, you need to look at the comments. You will find the script under 'samplescripts/Analog-Input' in the source tree. Based on the same mechanism (but quite different code inside the script) there is also a nice link:https://www.lesbonscomptes.com/upmpdcli/github-issues/upmpdcli-html/issue-71.html[HOWTO] explaining how to link a Bluetooth device to Songcast. The general idea is that upmpdcli will create an Openhome Source for each script it finds inside a designated directory. By default, the directory is +/usr/share/upmpdcli/src_scripts+, but it can be changed by setting the +ohsrc_scripts_dir+ configuration variable inside +/etc/upmpdcli.conf+ For managing analog sources, entries inside the directory will typically be created as symbolic links to the sample script, which is installed as +/usr/share/upmpdcli/Analog-Input+. For doing other things, the scripts will not be links, but they must still be named a bit specially. The scripts (or links) must be named like _SourceType_-_SourceName_, where _SourceType_ MUST BE one of +Analog+, +Digital+, or +Hdmi+, (which are all the same, and for display purpose only), and you can choose _SourceName_ as you wish, but it should contain no space characters. The Source will appear with type +Analog+, +Digital+ or +Hdmi+ and name _SourceName_ in an OpenHome Source select dialog (e.g from *upplay*). The +Analog-Input+ sample script has provisions for reading the device name and setup/teardown instructions from external files, so that it does not need to be modified. This is only relevant for the sample, please have a look at the comments for details. [[COMMAND-ENVIRON]] == Annex: command line and environment In most situations, *upmpdcli* will be run as follows: upmpdcli -D -c /etc/upmpdcli.conf The `-D` option tells *upmpdcli* to fork and run in background. The `-c` option specifies a configuration file. See the upmpdcli(1) manual page for more information about the command line. Some of the essential configuration parameters can be set from the command line, a configuration file, or the environment in this order of priority. It would be rather confusing to use a mix of methods, so you should probably chose one. All parameters have defaults, and a minimal installation will need no customisation at all. |======================== |What|Command line|Environment|Config variable |Configuration file name|-c config|$UPMPD_CONFIG| |Host name or IP address where *MPD* runs|-h mpdhost|$UPMPD_HOST|mpdhost |TCP port for *MPD*|-p mpdport|$UPMPD_PORT|mpdport |UPnP "friendly name" for the device. This gets displayed in network search results.|-f friendlyname|$UPMPD_FRIENDLYNAME|friendlyname |Log file name. Leave empty for stderr|-d logfilename||logfilename |Verbosity level (0-4)|-l loglevel||loglevel |UPnP network interface|-i iface|$UPMPD_UPNPIFACE|upnpiface |UPnP port|-P port|$UPMPD_UPNPPORT|upnpport |=========================== [[UPMPDCLI-CONFIG-GUI]] == Annex: the upmpdcli-config GUI configuration editor. The upmdpcli-config tool presents a graphical interface for creating an upmpdcli configuration file. It presents different tabs with controls, directly matching the organisation of the parameters in the documentation. Both the tool and the manual are generated from data in the reference configuration distributed with the package (typically '/usr/share/upmpdcli/upmpdcli.conf-dist'). You can use the tool to create a configuration from scratch (just tweak what you want, then save the file), or for editing an existing configuration. The latter can be done either by passing the configuration file name on the command line, or by using the 'Open' menu entry. If you use the command line, the parameter automatically becomes the target of a future 'Save' command (you can change it by using 'Save as'). When using 'Open', you will be asked to choose an output destination when saving. Each control has a label, and an longer explanation in a tooltip, which you can see by leaving the cursor on the label. Once you are satisfied with the changes, you can save the file, and move it to its normal location if needed (e.g. '/etc/upmpdcli.conf'). Only the parameters which were changed in the configuration you edit (if not starting from scratched), and those which you change through the interface, will be saved to the output. Parameters unchanged from the defaults will not appear at all, allowing for a less cluttered file. [[SCWEB]] == Annex: the scweb Songcast control interface. This is only useful if you have no machine on which to run *upplay*, or *Bubble DS Next*. Either are easier to use than the web tool. To avoid having to access the command line interface to control the *Songcast* groups, *upmpdcli* comes with a small Web server which uses *scctl* to actually do the work. This is found inside the `web/` subdirectory inside the *upmpdcli* source tree. The server is based on the link:http://bottlepy.org/docs/dev/index.html[Bottle Python Web Framework] and it only depends on Python (version 2 and 3 are supported by *Bottle*, but the current app only works with Python 2). You can use the `scweb-standalone.py` script to manually start the interface: python2 ./scweb-standalone.py This will start a server on localhost, on port 8680 by default which is good for testing, but not very useful. Use the -a 0.0.0.0 option to let the server answer on all local addresses (or specify the address to use a specific interface): python2 ./scweb-standalone.py -a 0.0.0.0 -p can be used to specify a port. Once started, connecting to the server from any browser should hopefully display a reasonably self-explanatory interface. Recent *upmpdcli* packages install the web app as a service named *scweb*. The service is not started by default though, you need to edit `/etc/default/scweb`.